Skip to content

Model for formula_energy in ssub database

  • Description: This is a benchmark to evaluate how accurately an AI model can predict the experimental formation energy from chemical formula using the SGTE Solid SUBstance (SSUB) database. Here we use mean absolute error (MAE) to compare models with respect to experimental accuracy. External links: https://materials.springer.com/lb/docs/sm_lbs_978-3-540-45280-5_1


Reference(s): https://www.nature.com/articles/s41598-018-35934-y, https://doi.org/10.1103/PhysRevMaterials.2.083801, https://hackingmaterials.lbl.gov/matminer/, https://www.nature.com/articles/s41524-023-01012-9;https://hackingmaterials.lbl.gov/matminer, https://epubs.siam.org/doi/abs/10.1137/1.9781611977172.39, https://dl.acm.org/doi/abs/10.1145/3292500.3330703, https://doi.org/10.1007/978-3-540-45280-5_1

Model benchmarks

Model nameDataset MAE Team name Dataset size Date submitted Notes
cfid_chemssub0.1694JARVIS172601-14-2023CSV, JSON, run.sh, Info
matminer_lgbmssub0.1441Matminer172601-14-2023CSV, JSON, run.sh, Info
BRNetssub0.1143NorthWestern_University172601-14-2023CSV, JSON, run.sh, Info
ElemNet2_TLssub0.0924NorthWestern_University172601-14-2023CSV, JSON, run.sh, Info
ElemNet2_SCssub0.2064NorthWestern_University172601-14-2023CSV, JSON, run.sh, Info
matminer_xgboostssub0.1244UofT172605-22-2023CSV, JSON, run.sh, Info
ElemNet1ssub0.2068NorthWestern_University172601-14-2023CSV, JSON, run.sh, Info
matminer_rfssub0.1468UofT172605-22-2023CSV, JSON, run.sh, Info
IRNet_EFssub0.1319NorthWestern_University172601-14-2023CSV, JSON, run.sh, Info
element_fraction_descssub0.3807JARVIS172601-14-2023CSV, JSON, run.sh, Info
BNetssub0.1286NorthWestern_University172601-14-2023CSV, JSON, run.sh, Info