Model for formula_energy in ssub database¶
- Description: This is a benchmark to evaluate how accurately an AI model can predict the experimental formation energy from chemical formula using the SGTE Solid SUBstance (SSUB) database. Here we use mean absolute error (MAE) to compare models with respect to experimental accuracy. External links: https://materials.springer.com/lb/docs/sm_lbs_978-3-540-45280-5_1
Reference(s): https://doi.org/10.1103/PhysRevMaterials.2.083801, https://epubs.siam.org/doi/abs/10.1137/1.9781611977172.39, https://hackingmaterials.lbl.gov/matminer/, https://www.nature.com/articles/s41598-018-35934-y, https://dl.acm.org/doi/abs/10.1145/3292500.3330703, https://doi.org/10.1007/978-3-540-45280-5_1, https://www.nature.com/articles/s41524-023-01012-9;https://hackingmaterials.lbl.gov/matminer
Model benchmarks
| Model name | Dataset | MAE | Team name | Dataset size | Date submitted | Notes |
|---|---|---|---|---|---|---|
| matminer_rf | ssub | 0.1468 | UofT | 1726 | 05-22-2023 | CSV, JSON, run.sh, Info |
| matminer_lgbm | ssub | 0.1441 | Matminer | 1726 | 01-14-2023 | CSV, JSON, run.sh, Info |
| BRNet | ssub | 0.1143 | NorthWestern_University | 1726 | 01-14-2023 | CSV, JSON, run.sh, Info |
| cfid_chem | ssub | 0.1694 | JARVIS | 1726 | 01-14-2023 | CSV, JSON, run.sh, Info |
| element_fraction_desc | ssub | 0.3807 | JARVIS | 1726 | 01-14-2023 | CSV, JSON, run.sh, Info |
| ElemNet2_SC | ssub | 0.2064 | NorthWestern_University | 1726 | 01-14-2023 | CSV, JSON, run.sh, Info |
| ElemNet1 | ssub | 0.2068 | NorthWestern_University | 1726 | 01-14-2023 | CSV, JSON, run.sh, Info |
| BNet | ssub | 0.1286 | NorthWestern_University | 1726 | 01-14-2023 | CSV, JSON, run.sh, Info |
| matminer_xgboost | ssub | 0.1244 | UofT | 1726 | 05-22-2023 | CSV, JSON, run.sh, Info |
| IRNet_EF | ssub | 0.1319 | NorthWestern_University | 1726 | 01-14-2023 | CSV, JSON, run.sh, Info |
| ElemNet2_TL | ssub | 0.0924 | NorthWestern_University | 1726 | 01-14-2023 | CSV, JSON, run.sh, Info |