Explore State-of-the-Art Materials Design Methods
Table of Contents
Introduction¶
This project provides benchmark-performances of various methods for materials science applications using the datasets available in JARVIS-Tools databases. Some of the categories are: Artificial Intelligence (AI), Electronic Structure (ES), Force-field (FF), Quantum Computation (QC) and Experiments (EXP). There are a variety of properties included in the benchmark. In addition to prediction results, we attempt to capture the underlyig software, hardware and instrumental frameworks to enhance reproducibility. This project is a part of the NIST-JARVIS infrastructure.
Usually, codes are kept in GitHub/GitLab etc., data is kept in Zenodo/Figshare/NIST Materials data etc., we recommend keeping the benchmarks in the JARVIS-Leaderboard to enhance reproducibility and transparency.
- Number of benchmarks: 297
- Number of contributions: 1671
- Number of datapoints: 8745168
A brief summary table is given below:
Category/Sub-cat. | SinglePropertyPrediction | SinglePropertyClass | MLFF | TextClass | TokenClass | TextSummary | TextGen | AtomGen | ImageClass | Spectra | EigenSolver |
AI | 441 | 21 | 116 | 28 | 1 | 1 | 3 | 3 | 2 | 1 | - |
ES | 732 | - | - | - | - | - | - | - | - | 10 | - |
FF | 282 | - | - | - | - | - | - | - | - | - | - |
QC | - | - | - | - | - | - | - | - | - | - | 6 |
EXP | 7 | - | - | - | - | - | - | - | - | 18 | - |
Example benchmarks¶
Click on the entries in the Benchmark column. You'll be able to see performance comparison, methods available for each benchmark, CSV file submitted for the contribution, JSON file for ground trutch data, run.sh script for running the method and Info for metadata associated with the method.
Category | Sub-category | Benchmark | Method | Metric | Score | Team | Dataset | Size |
---|---|---|---|---|---|---|---|---|
AI | SinglePropertyPrediction | ssub_formula_energy | ElemNet2_TL | MAE | 0.0924 | NorthWestern_University | ssub | 1726 |
AI | SinglePropertyPrediction | dft_3d_formation_energy_peratom | kgcnn_coGN | MAE | 0.0271 | kgcnn | dft_3d | 55713 |
AI | SinglePropertyPrediction | dft_3d_optb88vdw_bandgap | kgcnn_coGN | MAE | 0.1219 | kgcnn | dft_3d | 55713 |
AI | SinglePropertyPrediction | dft_3d_optb88vdw_total_energy | kgcnn_coGN | MAE | 0.0262 | kgcnn | dft_3d | 55713 |
AI | SinglePropertyPrediction | dft_3d_bulk_modulus_kv | kgcnn_coNGN | MAE | 8.7022 | kgcnn | dft_3d | 19680 |
AI | SinglePropertyClass | dft_3d_optb88vdw_bandgap | matminer_xgboost | ACC | 0.9361 | UofT | dft_3d | 55713 |
AI | SinglePropertyPrediction | qm9_std_jctc_LUMO | alignn_model | MAE | 0.0175 | ALIGNN | qm9_std_jctc | 130829 |
AI | SinglePropertyPrediction | hmof_max_co2_adsp | matminer_xgboost | MAE | 0.4622 | UofT | hmof | 137651 |
AI | MLFF | alignn_ff_db_energy | alignnff_pretrained_wt0.1 | MAE | 0.0342 | JARVIS | alignn_ff_db | 307111 |
AI | MLFF | mlearn_Si_forces | alignnff_mlearn_wt1 | MULTIMAE | 0.06942387617720659 | JARVIS | mlearn_Si | 239 |
AI | ImageClass | stem_2d_image_bravais_class | densenet_model | ACC | 0.8304 | JARVIS | stem_2d_image | 9150 |
AI | TextClass | arXiv_categories | svc_model_text_title_abstract | ACC | 0.9082 | ChemNLP | arXiv | 100994 |
FF | SinglePropertyPrediction | dft_3d_bulk_modulus_JVASP_816_Al | 2017--Kim-J-S--Pt-Al--LAMMPS--ipr1 | MAE | 0.0114 | IPR | dft_3d | 1 |
ES | SinglePropertyPrediction | dft_3d_bulk_modulus_JVASP_816_Al | vasp_optcx13 | MAE | 0.2 | JARVIS | dft_3d | 1 |
ES | SinglePropertyPrediction | dft_3d_bulk_modulus | vasp_opt86b | MAE | 4.6619 | JARVIS | dft_3d | 21 |
ES | SinglePropertyPrediction | dft_3d_bulk_modulus_JVASP_1002_Si | vasp_scan | MAE | 0.669 | JARVIS | dft_3d | 1 |
ES | SinglePropertyPrediction | dft_3d_bandgap | vasp_tbmbj | MAE | 0.4981 | JARVIS | dft_3d | 54 |
ES | SinglePropertyPrediction | dft_3d_bandgap_JVASP_1002_Si | gpaw_gllbsc | MAE | 0.0048 | GPAW | dft_3d | 1 |
ES | SinglePropertyPrediction | dft_3d_epsx | vasp_optb88vdw_linopt | MAE | 1.4638 | JARVIS | dft_3d | 16 |
ES | SinglePropertyPrediction | dft_3d_Tc_supercon_JVASP_1151_MgB2 | qe_pbesol_gbrv | MAE | 6.3148 | JARVIS | dft_3d | 1 |
ES | SinglePropertyPrediction | dft_3d_Tc_supercon | qe_pbesol_gbrv | MAE | 3.3785 | JARVIS | dft_3d | 14 |
ES | SinglePropertyPrediction | dft_3d_slme | vasp_tbmbj | MAE | 5.0925 | JARVIS | dft_3d | 5 |
ES | Spectra | dft_3d_dielectric_function | vasp_tbmbj | MULTIMAE | 2.8799621766740207 | JARVIS | dft_3d | 4 |
QC | EigenSolver | dft_3d_electron_bands_JVASP_816_Al_WTBH | qiskit_vqd_SU2_c6 | MULTIMAE | 0.002963733593749998 | JARVIS | dft_3d | 1 |
EXP | Spectra | dft_3d_XRD_JVASP_19821_MgB2 | bruker_d8 | MULTIMAE | 0.020040003548149166 | MML-BrukerD8 | dft_3d | 1 |
EXP | Spectra | nist_isodb_co2_RM_8852 | 10.1007s10450-018-9958-x.Lab01 | MULTIMAE | 0.02129168060976213 | FACTlab | nist_isodb | 1 |
Help¶
If you have a question/suggestion, raise a GitHub issue or submit a Google form request.
License¶
This template is served under the NIST license.
Read the LICENSE file for more info.