Model for arXiv text class¶
- Description: This is an AI benchmark to evaluate how accurately text data is classified into different categories, using the arXiv dataset. Here we use accuracy of classification (ACC) to compare how well each model classifies the text data, comparing to the ground truth classification of the arXiv categories.
Reference(s): https://doi.org/10.48550/arXiv.1905.00075, https://github.com/usnistgov/chemnlp
Model benchmarks
| Model name | Dataset | Accuracy | Team name | Dataset size | Date submitted | Notes |
|---|---|---|---|---|---|---|
| random_forest_text_title_abstract | arXiv | 0.8854 | ChemNLP | 100994 | 01-14-2023 | CSV, JSON, run.sh, Info |
| svc_model_text_abstract | arXiv | 0.9082 | ChemNLP | 100994 | 01-14-2023 | CSV, JSON, run.sh, Info |
| random_forest_text_title | arXiv | 0.8681 | ChemNLP | 100994 | 01-14-2023 | CSV, JSON, run.sh, Info |
| svc_model_text_title | arXiv | 0.8469 | ChemNLP | 100994 | 01-14-2023 | CSV, JSON, run.sh, Info |
| logisticreg_model_text_abstract | arXiv | 0.8543 | ChemNLP | 100994 | 01-14-2023 | CSV, JSON, run.sh, Info |
| logisticreg_model_text_title_abstract | arXiv | 0.8597 | ChemNLP | 100994 | 01-14-2023 | CSV, JSON, run.sh, Info |
| random_forest_text_abstract | arXiv | 0.884 | ChemNLP | 100994 | 01-14-2023 | CSV, JSON, run.sh, Info |
| svc_model_text_title_abstract | arXiv | 0.9082 | ChemNLP | 100994 | 01-14-2023 | CSV, JSON, run.sh, Info |
| logisticreg_model_text_title | arXiv | 0.7903 | ChemNLP | 100994 | 01-14-2023 | CSV, JSON, run.sh, Info |