Model for elastic tensor (C11) of CHIPSFF dataset¶
- Description: This is a benchmark to evaluate how accurately an AI model can predict the elastic tensor (C11) using the CHIPSFF subset of JARVIS-DFT (dft_3d) dataset. The dataset contains different types of chemical formula and atomic structures of materials common to semiconductor device components. Here we use mean absolute error (MAE) to compare models with respect to DFT accuracy.
Reference(s): https://www.nature.com/articles/s41524-020-00440-1, https://github.com/usnistgov/chipsff, https://doi.org/10.48550/arXiv.2305.11842
Model benchmarks
| Model name | Dataset | ACC | Team name | Dataset size | Date submitted | Notes |
|---|---|---|---|---|---|---|
| eqV2_31M_omat_mp_salex | dft_3d_chipsff | 45.6873 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| mace-alexandria | dft_3d_chipsff | 69.397 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| sevennet | dft_3d_chipsff | 42.4671 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| matgl-direct | dft_3d_chipsff | 72.1509 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| mace | dft_3d_chipsff | 42.3521 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| orb-d3-v2 | dft_3d_chipsff | 76.9822 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| chgnet | dft_3d_chipsff | 59.3418 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| orb-v2 | dft_3d_chipsff | 80.6045 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| alignn_ff | dft_3d_chipsff | 196.1943 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| mattersim | dft_3d_chipsff | 31.8093 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| eqV2_153M_omat | dft_3d_chipsff | 74.7347 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| eqV2_86M_omat_mp_salex | dft_3d_chipsff | 43.3645 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| matgl | dft_3d_chipsff | 63.5102 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| eqV2_86M_omat | dft_3d_chipsff | 86.0911 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| mace-mpa | dft_3d_chipsff | 35.2215 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |
| eqV2_31M_omat | dft_3d_chipsff | 78.2688 | JARVIS | 104 | 3-11-2025 | CSV, JSON, run.sh, Info |