- Description: This is a benchmark to evaluate how accurately an AI model can predict the energy of adsorption of N using the TinNet dataset. The dataset contains different types of heterogeneous catalysts. Here we use mean absolute error (MAE) to compare models with respect to DFT accuracy. External links: https://www.nature.com/articles/s41467-021-25639-8
Reference(s): https://doi.org/10.1038/s41467-021-25639-8, https://pubs.rsc.org/en/content/articlehtml/2023/dd/d2dd00096b, https://www.nature.com/articles/s41524-021-00650-1
Model benchmarks
| Model name | Dataset |
MAE |
Team name |
Dataset size |
Date submitted |
Notes |
| alignn_model | tinnet_N | 0.0871 | ALIGNN | 327 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_COOH-test-mae.csv.zip | tinnet_N | 3.8475 | ALIGNN | 327 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_CHO-test-mae.csv.zip | tinnet_N | 1.4925 | ALIGNN | 327 | 01-14-2023 | CSV, JSON, run.sh, Info |
| CHGNet_pretrained | tinnet_N | 11.1922 | ALIGNN | 327 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_O-test-mae.csv.zip | tinnet_N | 5.6385 | ALIGNN | 327 | 01-14-2023 | CSV, JSON, run.sh, Info |
| MATGL_pretrained | tinnet_N | 10.7449 | ALIGNN | 327 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_OH-test-mae.csv.zip | tinnet_N | 5.3781 | ALIGNN | 327 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-tinnet_N-test-mae.csv.zip | tinnet_N | 0.1283 | ALIGNN | 327 | 01-14-2023 | CSV, JSON, run.sh, Info |
| MACE_pretrained | tinnet_N | 10.999 | ALIGNN | 327 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-tinnet_OH-test-mae.csv.zip | tinnet_N | 2.2395 | ALIGNN | 327 | 01-14-2023 | CSV, JSON, run.sh, Info |
| alignnff_pretrained_wt0.1 | tinnet_N | 8.2278 | JARVIS | 327 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_CO-test-mae.csv.zip | tinnet_N | 2.6554 | ALIGNN | 327 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-tinnet_O-test-mae.csv.zip | tinnet_N | 0.871 | ALIGNN | 327 | 01-14-2023 | CSV, JSON, run.sh, Info |