Model for arXiv text class¶
- Description: This is an AI benchmark to evaluate how accurately text data is classified into different categories, using the PubChem dataset. Here we use accuracy of classification (ACC) to compare how well each model classifies the text data, comparing to the ground truth classification of the PubChem categories.
Reference(s): https://github.com/usnistgov/chemnlp, https://doi.org/10.1093/nar/gkaa971
Model benchmarks
| Model name | Dataset | Accuracy | Team name | Dataset size | Date submitted | Notes |
|---|---|---|---|---|---|---|
| logisticreg_model_text_title | pubchem | 0.9206 | ChemNLP | 44500 | 01-14-2023 | CSV, JSON, run.sh, Info |
| logisticreg_model_text_abstract | pubchem | 0.9276 | ChemNLP | 44500 | 01-14-2023 | CSV, JSON, run.sh, Info |
| random_forest_text_title | pubchem | 0.9449 | ChemNLP | 44500 | 01-14-2023 | CSV, JSON, run.sh, Info |
| svc_model_text_title | pubchem | 0.9458 | ChemNLP | 44500 | 01-14-2023 | CSV, JSON, run.sh, Info |
| svc_model_text_title_abstract | pubchem | 0.94 | ChemNLP | 44500 | 01-14-2023 | CSV, JSON, run.sh, Info |
| random_forest_text_abstract | pubchem | 0.9317 | ChemNLP | 44500 | 01-14-2023 | CSV, JSON, run.sh, Info |
| svc_model_text_abstract | pubchem | 0.94 | ChemNLP | 44500 | 01-14-2023 | CSV, JSON, run.sh, Info |
| logisticreg_model_text_title_abstract | pubchem | 0.9674 | ChemNLP | 44500 | 01-14-2023 | CSV, JSON, run.sh, Info |
| random_forest_text_title_abstract | pubchem | 0.9674 | ChemNLP | 44500 | 01-14-2023 | CSV, JSON, run.sh, Info |