Results¶

Jobs Table¶

Contains the following fields:

• Team : The team name you selected.

• Execution Timestamp : When your submission was received by the test and evaluation server and added to the input queue.

• Job Status : “None” if there is no active job on the server. Otherwise it will indicate the jobs status from the queue manager. or “Ok”. File status will show “Multiple Files Shared” if you share multiple files with the same name with the TrojAI Drive Account.

• File Status : “None” if no files are shared with the TrojAI Drive Account. Ok” if one file per server is found. “Multiple Files Shared” if there are multiple files shared with the TrojAI Drive Account.

• File Timestamp : The timestamp of the file received from your team email which caused a job to be created, entering your submission into the work queue. This timestamp might change if you changed the file while your job was in the input queue.

• Time until Next Execution : How long before your next submission will be eligible for entry into the work queue.

Results Table¶

This table shows the single submission with the lowest Cross Entropy score per team and contains the following fields:

• Team : The team name you selected.

• Cross Entropy : Your Cross Entropy loss for a given submission.

• CE 95% CI : 95% Confidence Interval for your Cross Entropy loss for a given submission.

• Brier Score : Your Brier Score for a given submission.

• ROC-AUC : Your Retriever Operator Characteristic Area Under the Curve for a given submission.

• Runtime (s) : Your submission wall runtime in seconds.

• Execution Timestamp : When the container was executed on the server.

• File Timestamp : The Google Drive file modified date of the container executed on the server.

• Parsing Errors : Any errors which stem from parsing your output results files.

• Launch Errors : Any errors which stem from running your container.

All Results Table¶

This table show the all submission results and contains the following fields:

Error Codes¶

The leaderboard has a section at the bottom indicating all the errors the test and evaluation server can encounter when running your container.

Previous Leaderboards¶

The Cross Entropy loss values (and confidence intervals) reported by the TrojAI leaderboard are only indicative of trojan detector performance on the specific dataset the detector was evaluated on. The TrojAI leaderboard results do not indicate general purpose trojan detection algorithm quality.