Model for surface energy of CHIPSFF dataset¶
- Description: This is a benchmark to evaluate how accurately an AI model can predict the surface energy (J/m^2) using the CHIPSFF subset of JARVIS-DFT (dft_3d) dataset. The dataset contains different types of chemical formula and atomic structures of materials common to semiconductor device components. Here we use mean absolute error (MAE) to compare models with respect to DFT accuracy.
Reference(s): https://github.com/usnistgov/chipsff, https://doi.org/10.48550/arXiv.2305.11842, https://www.nature.com/articles/s41524-020-00440-1
Model benchmarks
| Model name | Dataset | ACC | Team name | Dataset size | Date submitted | Notes |
|---|---|---|---|---|---|---|
| matgl | dft_3d_chipsff | 0.8858 | JARVIS | 82 | 11-01-2024 | CSV, JSON, run.sh, Info |
| eqV2_31M_omat | dft_3d_chipsff | 0.3199 | JARVIS | 82 | 11-01-2024 | CSV, JSON, run.sh, Info |
| eqV2_153M_omat | dft_3d_chipsff | 0.3261 | JARVIS | 82 | 11-01-2024 | CSV, JSON, run.sh, Info |
| eqV2_86M_omat | dft_3d_chipsff | 0.3652 | JARVIS | 82 | 11-01-2024 | CSV, JSON, run.sh, Info |
| mace-alexandria | dft_3d_chipsff | 0.6939 | JARVIS | 82 | 11-01-2024 | CSV, JSON, run.sh, Info |
| orb-v2 | dft_3d_chipsff | 0.245 | JARVIS | 82 | 11-01-2024 | CSV, JSON, run.sh, Info |
| chgnet | dft_3d_chipsff | 0.665 | JARVIS | 82 | 11-01-2024 | CSV, JSON, run.sh, Info |
| mace | dft_3d_chipsff | 0.4617 | JARVIS | 82 | 11-01-2024 | CSV, JSON, run.sh, Info |
| matgl-direct | dft_3d_chipsff | 0.713 | JARVIS | 82 | 11-01-2024 | CSV, JSON, run.sh, Info |
| eqV2_31M_omat_mp_salex | dft_3d_chipsff | 0.2144 | JARVIS | 82 | 11-01-2024 | CSV, JSON, run.sh, Info |
| alignn_ff_5_27_24 | dft_3d_chipsff | 1.4826 | JARVIS | 82 | 11-01-2024 | CSV, JSON, run.sh, Info |
| alignn_ff_12_2_24 | dft_3d_chipsff | 1.9218 | JARVIS | 82 | 11-01-2024 | CSV, JSON, run.sh, Info |
| sevennet | dft_3d_chipsff | 0.3319 | JARVIS | 82 | 11-01-2024 | CSV, JSON, run.sh, Info |
| eqV2_86M_omat_mp_salex | dft_3d_chipsff | 0.2141 | JARVIS | 82 | 11-01-2024 | CSV, JSON, run.sh, Info |