Model for superconducting Tc using chemical formula¶
- Description: This is a benchmark to evaluate how accurately an AI model can predict the transition temperature (Tc) of a superconductor from chemical formula using the supercon dataset, which only contains Tc and chemical formula. Here we use mean absolute error (MAE) to compare models with respect to experimental accuracy.
Reference(s): https://doi.org/10.1103/PhysRevMaterials.2.083801, https://epubs.siam.org/doi/abs/10.1137/1.9781611977172.39, https://hackingmaterials.lbl.gov/matminer/, https://www.nature.com/articles/s41598-018-35934-y, https://dl.acm.org/doi/abs/10.1145/3292500.3330703, https://doi.org/10.1038/s41524-018-0085-8, https://www.nature.com/articles/s41524-023-01012-9;https://hackingmaterials.lbl.gov/matminer
Model benchmarks
| Model name | Dataset | MAE | Team name | Dataset size | Date submitted | Notes |
|---|---|---|---|---|---|---|
| matminer_rf | supercon_chem | 4.8511 | UofT | 16414 | 05-22-2023 | CSV, JSON, run.sh, Info |
| matminer_lgbm | supercon_chem | 5.1519 | Matminer | 16414 | 01-14-2023 | CSV, JSON, run.sh, Info |
| BRNet | supercon_chem | 5.437 | NorthWestern_University | 16414 | 01-14-2023 | CSV, JSON, run.sh, Info |
| cfid_chem | supercon_chem | 5.0326 | JARVIS | 16414 | 01-14-2023 | CSV, JSON, run.sh, Info |
| element_fraction_desc | supercon_chem | 5.2921 | JARVIS | 16414 | 01-14-2023 | CSV, JSON, run.sh, Info |
| ElemNet2_SC | supercon_chem | 5.114 | NorthWestern_University | 16414 | 01-14-2023 | CSV, JSON, run.sh, Info |
| ElemNet1 | supercon_chem | 5.5901 | NorthWestern_University | 16414 | 01-14-2023 | CSV, JSON, run.sh, Info |
| BNet | supercon_chem | 5.3076 | NorthWestern_University | 16414 | 01-14-2023 | CSV, JSON, run.sh, Info |
| matminer_xgboost | supercon_chem | 4.9295 | UofT | 16414 | 05-22-2023 | CSV, JSON, run.sh, Info |
| IRNet_EF | supercon_chem | 5.835 | NorthWestern_University | 16414 | 01-14-2023 | CSV, JSON, run.sh, Info |
| ElemNet2_TL | supercon_chem | 5.4937 | NorthWestern_University | 16414 | 01-14-2023 | CSV, JSON, run.sh, Info |