Model for bulk modulus (Kv) of CHIPSFF dataset¶
- Description: This is a benchmark to evaluate how accurately an AI model can predict the bulk modulus (Kv) using the CHIPSFF subset of JARVIS-DFT (dft_3d) dataset. The dataset contains different types of chemical formula and atomic structures of materials common to semiconductor device components. Here we use mean absolute error (MAE) to compare models with respect to DFT accuracy.
Reference(s): https://www.nature.com/articles/s41524-020-00440-1, https://github.com/usnistgov/chipsff, https://doi.org/10.48550/arXiv.2305.11842
Model benchmarks
| Model name | Dataset | ACC | Team name | Dataset size | Date submitted | Notes |
|---|---|---|---|---|---|---|
| eqV2_31M_omat_mp_salex | dft_3d_chipsff | 112.3685 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| mace-alexandria | dft_3d_chipsff | 77.9944 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| sevennet | dft_3d_chipsff | 103.6429 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| matgl-direct | dft_3d_chipsff | 83.7321 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| mace | dft_3d_chipsff | 94.196 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| orb-d3-v2 | dft_3d_chipsff | 130.4195 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| chgnet | dft_3d_chipsff | 85.5984 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| orb-v2 | dft_3d_chipsff | 118.2786 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| alignn_ff | dft_3d_chipsff | 135.9185 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| mattersim | dft_3d_chipsff | 110.1779 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| eqV2_153M_omat | dft_3d_chipsff | 117.1085 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| eqV2_86M_omat_mp_salex | dft_3d_chipsff | 113.7747 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| matgl | dft_3d_chipsff | 82.9773 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| eqV2_86M_omat | dft_3d_chipsff | 118.2267 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| mace-mpa | dft_3d_chipsff | 109.9045 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| eqV2_31M_omat | dft_3d_chipsff | 114.3849 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |