Explore State-of-the-Art Materials Design Methods
Table of Contents
Introduction¶
This project provides benchmark performances of various methods for materials science applications using the datasets available in JARVIS-Tools databases. Some of the categories are: Artificial Intelligence (AI), Electronic Structure (ES), Force-field (FF), Quantum Computation (QC) and Experiments (EXP). A variety of properties are included in the benchmark.
Typically, codes are kept in platforms like GitHub/GitLab, and data is stored in repositories like Zenodo/Figshare/NIST Materials Data. We recommend keeping the benchmarks in the JARVIS-Leaderboard to enhance reproducibility and transparency. In addition to prediction results, we aim to capture the underlying software, hardware, and instrumental frameworks to enhance reproducibility. This project is a part of the NIST-JARVIS infrastructure. As a minimum check, we test rebuilding of the leaderboard and installations of software using GitHub actions.
- Number of benchmarks: 322
- Number of contributions: 2073
- Number of datapoints: 8752824
A brief summary table is given below:
| Category/Sub-cat. | SinglePropertyPrediction | SinglePropertyClass | MLFF | TextClass | TokenClass | TextSummary | TextGen | AtomGen | ImageClass | Spectra | EigenSolver |
| AI | 686 | 21 | 272 | 28 | 1 | 1 | 3 | 5 | 2 | 1 | - |
| ES | 731 | - | - | - | - | - | - | - | - | 10 | - |
| FF | 282 | - | - | - | - | - | - | - | - | - | - |
| QC | - | - | - | - | - | - | - | - | - | - | 6 |
| EXP | 7 | - | - | - | - | - | - | - | - | 18 | - |
Example benchmarks¶
Click on the entries in the Benchmark column. You'll be able to see performance comparison, methods available for each benchmark, CSV file submitted for the contribution, JSON file for ground trutch data, run.sh script for running the method and Info for metadata associated with the method.
| Category | Sub-category | Benchmark | Method | Metric | Score | Team | Dataset | Size |
|---|---|---|---|---|---|---|---|---|
| AI | SinglePropertyPrediction | ssub_formula_energy | ElemNet2_TL | MAE | 0.0924 | NorthWestern_University | ssub | 1726 |
| AI | SinglePropertyPrediction | dft_3d_formation_energy_peratom | kgcnn_coGN | MAE | 0.0271 | kgcnn | dft_3d | 55713 |
| AI | SinglePropertyPrediction | dft_3d_optb88vdw_bandgap | kgcnn_coGN | MAE | 0.1219 | kgcnn | dft_3d | 55713 |
| AI | SinglePropertyPrediction | dft_3d_optb88vdw_total_energy | kgcnn_coGN | MAE | 0.0262 | kgcnn | dft_3d | 55713 |
| AI | SinglePropertyPrediction | dft_3d_bulk_modulus_kv | kgcnn_coNGN | MAE | 8.7022 | kgcnn | dft_3d | 19680 |
| AI | SinglePropertyClass | dft_3d_optb88vdw_bandgap | matminer_xgboost | ACC | 0.9361 | UofT | dft_3d | 55713 |
| AI | SinglePropertyPrediction | qm9_std_jctc_LUMO | alignn_model | MAE | 0.0175 | ALIGNN | qm9_std_jctc | 130829 |
| AI | SinglePropertyPrediction | hmof_max_co2_adsp | matminer_xgboost | MAE | 0.4622 | UofT | hmof | 137651 |
| AI | MLFF | alignn_ff_db_energy | alignnff_pretrained_wt0.1 | MAE | 0.0342 | JARVIS | alignn_ff_db | 307111 |
| AI | MLFF | mlearn_Si_forces | alignnff_mlearn_wt1 | MULTIMAE | 0.06942387617720659 | JARVIS | mlearn_Si | 239 |
| AI | ImageClass | stem_2d_image_bravais_class | densenet_model | ACC | 0.8304 | JARVIS | stem_2d_image | 9150 |
| AI | TextClass | arXiv_categories | svc_model_text_abstract | ACC | 0.9082 | ChemNLP | arXiv | 100994 |
| FF | SinglePropertyPrediction | dft_3d_bulk_modulus_JVASP_816_Al | 2017--Kim-J-S--Pt-Al--LAMMPS--ipr1 | MAE | 0.0114 | IPR | dft_3d | 1 |
| ES | SinglePropertyPrediction | dft_3d_bulk_modulus_JVASP_816_Al | vasp_optcx13 | MAE | 0.2 | JARVIS | dft_3d | 1 |
| ES | SinglePropertyPrediction | dft_3d_bulk_modulus | vasp_opt86b | MAE | 4.6619 | JARVIS | dft_3d | 21 |
| ES | SinglePropertyPrediction | dft_3d_bulk_modulus_JVASP_1002_Si | vasp_scan | MAE | 0.669 | JARVIS | dft_3d | 1 |
| ES | SinglePropertyPrediction | dft_3d_bandgap | vasp_tbmbj | MAE | 0.4981 | JARVIS | dft_3d | 54 |
| ES | SinglePropertyPrediction | dft_3d_bandgap_JVASP_1002_Si | gpaw_gllbsc | MAE | 0.0048 | GPAW | dft_3d | 1 |
| ES | SinglePropertyPrediction | dft_3d_epsx | vasp_optb88vdw_linopt | MAE | 1.4638 | JARVIS | dft_3d | 16 |
| ES | SinglePropertyPrediction | dft_3d_Tc_supercon_JVASP_1151_MgB2 | qe_pbesol_gbrv | MAE | 6.3148 | JARVIS | dft_3d | 1 |
| ES | SinglePropertyPrediction | dft_3d_Tc_supercon | qe_pbesol_gbrv | MAE | 3.3785 | JARVIS | dft_3d | 14 |
| ES | SinglePropertyPrediction | dft_3d_slme | vasp_tbmbj | MAE | 5.0925 | JARVIS | dft_3d | 5 |
| ES | Spectra | dft_3d_dielectric_function | vasp_tbmbj | MULTIMAE | 2.8799621766740207 | JARVIS | dft_3d | 4 |
| QC | EigenSolver | dft_3d_electron_bands_JVASP_816_Al_WTBH | qiskit_vqd_SU2_c6 | MULTIMAE | 0.002963733593749998 | JARVIS | dft_3d | 1 |
| EXP | Spectra | dft_3d_XRD_JVASP_19821_MgB2 | bruker_d8 | MULTIMAE | 0.020040003548149166 | MML-BrukerD8 | dft_3d | 1 |
| EXP | Spectra | nist_isodb_co2_RM_8852 | 10.1007s10450-018-9958-x.Lab01 | MULTIMAE | 0.02129168060976213 | FACTlab | nist_isodb | 1 |
Help¶
If you have a question/suggestion, raise a GitHub issue or submit a Google form request.
License¶
This template is served under the NIST license.
Read the LICENSE file for more info.