- Description: This is a benchmark to evaluate how accurately an AI model can predict the energy of adsorption of OH using the TinNet dataset. The dataset contains different types of heterogeneous catalysts. Here we use mean absolute error (MAE) to compare models with respect to DFT accuracy. External links: https://www.nature.com/articles/s41467-021-25639-8
Reference(s): https://pubs.rsc.org/en/content/articlehtml/2023/dd/d2dd00096b, https://www.nature.com/articles/s41524-021-00650-1, https://doi.org/10.1038/s41467-021-25639-8
Model benchmarks
| Model name | Dataset |
MAE |
Team name |
Dataset size |
Date submitted |
Notes |
| alignn_model | tinnet_OH | 0.0882 | ALIGNN | 746 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_COOH-test-mae.csv.zip | tinnet_OH | 1.9107 | ALIGNN | 746 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_CHO-test-mae.csv.zip | tinnet_OH | 1.0117 | ALIGNN | 746 | 01-14-2023 | CSV, JSON, run.sh, Info |
| CHGNet_pretrained | tinnet_OH | 16.1028 | ALIGNN | 746 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_O-test-mae.csv.zip | tinnet_OH | 3.6081 | ALIGNN | 746 | 01-14-2023 | CSV, JSON, run.sh, Info |
| MATGL_pretrained | tinnet_OH | 13.5468 | ALIGNN | 746 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_OH-test-mae.csv.zip | tinnet_OH | 3.4421 | ALIGNN | 746 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-tinnet_N-test-mae.csv.zip | tinnet_OH | 2.1692 | ALIGNN | 746 | 01-14-2023 | CSV, JSON, run.sh, Info |
| MACE_pretrained | tinnet_OH | 14.3575 | ALIGNN | 746 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-tinnet_OH-test-mae.csv.zip | tinnet_OH | 0.0843 | ALIGNN | 746 | 01-14-2023 | CSV, JSON, run.sh, Info |
| alignnff_pretrained_wt0.1 | tinnet_OH | 9.5889 | JARVIS | 746 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_CO-test-mae.csv.zip | tinnet_OH | 0.7498 | ALIGNN | 746 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-tinnet_O-test-mae.csv.zip | tinnet_OH | 1.8078 | ALIGNN | 746 | 01-14-2023 | CSV, JSON, run.sh, Info |