- Description: This is a benchmark to evaluate how accurately an AI model can predict the energy of adsorption of O using the TinNet dataset. The dataset contains different types of heterogeneous catalysts. Here we use mean absolute error (MAE) to compare models with respect to DFT accuracy. External links: https://www.nature.com/articles/s41467-021-25639-8
Reference(s): https://www.nature.com/articles/s41524-021-00650-1, https://doi.org/10.1038/s41467-021-25639-8, https://pubs.rsc.org/en/content/articlehtml/2023/dd/d2dd00096b
Model benchmarks
| Model name | Dataset |
MAE |
Team name |
Dataset size |
Date submitted |
Notes |
| MATGL_pretrained | tinnet_O | 4.9459 | ALIGNN | 745 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-tinnet_N-test-mae.csv.zip | tinnet_O | 1.1438 | ALIGNN | 745 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_O-test-mae.csv.zip | tinnet_O | 5.458 | ALIGNN | 745 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_OH-test-mae.csv.zip | tinnet_O | 5.2012 | ALIGNN | 745 | 01-14-2023 | CSV, JSON, run.sh, Info |
| alignn_model | tinnet_O | 0.3694 | ALIGNN | 745 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_CO-test-mae.csv.zip | tinnet_O | 2.4389 | ALIGNN | 745 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_COOH-test-mae.csv.zip | tinnet_O | 3.911 | ALIGNN | 745 | 01-14-2023 | CSV, JSON, run.sh, Info |
| alignnff_pretrained_wt0.1 | tinnet_O | 1.885 | JARVIS | 745 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-tinnet_O-test-mae.csv.zip | tinnet_O | 0.2657 | ALIGNN | 745 | 01-14-2023 | CSV, JSON, run.sh, Info |
| CHGNet_pretrained | tinnet_O | 6.0983 | ALIGNN | 745 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-tinnet_OH-test-mae.csv.zip | tinnet_O | 1.7819 | ALIGNN | 745 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_CHO-test-mae.csv.zip | tinnet_O | 2.0323 | ALIGNN | 745 | 01-14-2023 | CSV, JSON, run.sh, Info |
| MACE_pretrained | tinnet_O | 4.7917 | ALIGNN | 745 | 01-14-2023 | CSV, JSON, run.sh, Info |