- Description: This is a benchmark to evaluate how accurately an AI model can predict the energy of adsorption of O using the Automatic graph representation algorithm (AGRA) dataset. The dataset contains different types of heterogeneous catalysts. Here we use mean absolute error (MAE) to compare models with respect to DFT accuracy. External links: https://pubs.aip.org/aip/aml/article/1/3/036103/2900763/Automatic-graph-representation-algorithm-for
Reference(s): https://pubs.rsc.org/en/content/articlehtml/2023/dd/d2dd00096b, https://doi.org/10.1063/5.0140487, https://www.nature.com/articles/s41524-021-00650-1
Model benchmarks
| Model name | Dataset |
MAE |
Team name |
Dataset size |
Date submitted |
Notes |
| alignn_model | AGRA_O | 0.1134 | ALIGNN | 1000 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_COOH-test-mae.csv.zip | AGRA_O | 1.9278 | ALIGNN | 1000 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_CHO-test-mae.csv.zip | AGRA_O | 4.0692 | ALIGNN | 1000 | 01-14-2023 | CSV, JSON, run.sh, Info |
| CHGNet_pretrained | AGRA_O | 12.2868 | ALIGNN | 1000 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_O-test-mae.csv.zip | AGRA_O | 0.0319 | ALIGNN | 1000 | 01-14-2023 | CSV, JSON, run.sh, Info |
| MATGL_pretrained | AGRA_O | 11.5325 | ALIGNN | 1000 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_OH-test-mae.csv.zip | AGRA_O | 0.264 | ALIGNN | 1000 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-tinnet_N-test-mae.csv.zip | AGRA_O | 5.4815 | ALIGNN | 1000 | 01-14-2023 | CSV, JSON, run.sh, Info |
| MACE_pretrained | AGRA_O | 11.4508 | ALIGNN | 1000 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-tinnet_OH-test-mae.csv.zip | AGRA_O | 3.0833 | ALIGNN | 1000 | 01-14-2023 | CSV, JSON, run.sh, Info |
| alignnff_pretrained_wt0.1 | AGRA_O | 6.2945 | JARVIS | 1000 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-AGRA_CO-test-mae.csv.zip | AGRA_O | 3.0584 | ALIGNN | 1000 | 01-14-2023 | CSV, JSON, run.sh, Info |
| Out-AI-SinglePropertyPrediction-ead-tinnet_O-test-mae.csv.zip | AGRA_O | 4.8292 | ALIGNN | 1000 | 01-14-2023 | CSV, JSON, run.sh, Info |