Runs - Clinical Trials 2021¶

BARTRM3Filt¶

Run ID: BARTRM3Filt
Participant: ims_unipd
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 2048c029831c3a6bbf6e1f83abb2217b
Run description: BM25 + BART embeddings + RM3 pseudo RF

baseline¶

Run ID: baseline
Participant: DOSSIER
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: bc0fb691792603337c3dc2ee6131e440
Run description: bm25 baseline

bm25_text¶

Results | Participants | Input | Summary | Appendix

Run ID: bm25_text
Participant: HEG_Geneva
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: f225bc74f0f117cd2be9dce6fd650723
Run description: BM25 model of the Elasticsearch on multiple CT fields with eligibility filtering.

bm25_txt_map¶

Results | Participants | Input | Summary | Appendix

Run ID: bm25_txt_map
Participant: HEG_Geneva
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: f3b34b044059b73e7f3d53dd8abd8d9b
Run description: RRF combination of two BM25 models, one based on the original query texts, one based on query expansion by the Metamap framework

CincyMedIR_1¶

Run ID: CincyMedIR_1
Participant: CincyMedIR
Track: Clinical Trials
Year: 2021
Submission: 8/16/2021
Type: automatic
Task: primary
MD5: af389386534b7ccfdc888ef86889ee44
Run description: Elasticsearch Match topic text against brief title, official title, brief summary, detailed description Filter by age, gender, recruiting status, exclusion criteria

CincyMedIR_2¶

Run ID: CincyMedIR_2
Participant: CincyMedIR
Track: Clinical Trials
Year: 2021
Submission: 8/16/2021
Type: automatic
Task: primary
MD5: 53f0c0c8ce7a2a81db8bf723474a8bc5
Run description: Elasticsearch Match topic text against brief title, official title, brief summary, detailed description Filter by age, gender, recruiting status Rerank by exclusion criteria

CincyMedIR_3¶

Run ID: CincyMedIR_3
Participant: CincyMedIR
Track: Clinical Trials
Year: 2021
Submission: 8/16/2021
Type: automatic
Task: primary
MD5: bcf25d8c2b93ce036edd71c6a2bdbdd9
Run description: Elasticsearch Match topic text against brief title, official title, brief summary, detailed description Filter by age, gender, exclusion criteria Rerank by recruiting status

CincyMedIR_4¶

Run ID: CincyMedIR_4
Participant: CincyMedIR
Track: Clinical Trials
Year: 2021
Submission: 8/16/2021
Type: automatic
Task: primary
MD5: 8d53e76d458b8bb8ee91ceadea7bcd4f
Run description: Elasticsearch Match topic text against brief title, official title, brief summary, detailed description Filter by age, gender Rerank by exclusion criteria first, and then recruiting status

CincyMedIR_5¶

Run ID: CincyMedIR_5
Participant: CincyMedIR
Track: Clinical Trials
Year: 2021
Submission: 8/16/2021
Type: automatic
Task: primary
MD5: 76c95ee3796f3c8cefa4bceaba3cc490
Run description: Elasticsearch Match topic text against brief title, official title, brief summary, detailed description Filter by age, gender Rerank by recruiting status first, and then exclusion criteria

CSIROmed_abs¶

Run ID: CSIROmed_abs
Participant: CSIROmed
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: e9b9bd889b9c710ab883a822a83c36f7
Run description: Lucene's DFR + BERT-based reranking with titles and summaries.

CSIROmed_brd¶

Run ID: CSIROmed_brd
Participant: CSIROmed
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 03f278ab308bf1ab74319d20445578c9
Run description: Borda rank over other runs

CSIROmed_DCT¶

Run ID: CSIROmed_DCT
Participant: CSIROmed
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 7ceee1d03affb0dbd2a0ea4e922f211e
Run description: Lucene's DFR + DeepCT

CSIROmed_inc¶

Run ID: CSIROmed_inc
Participant: CSIROmed
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: d3107b739e251f4c96663900efb09ca5
Run description: Lucene's DFR + BERT-based reranking with titles and inclusion criteria

CSIROmedNIR¶

Run ID: CSIROmedNIR
Participant: CSIROmed
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: eac646e4e53ea1046ae0c58c502587b9
Run description: NIR (w. ES BM25)

CuiInc¶

Run ID: CuiInc
Participant: GU_BioMed
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: ac39e10c5605f6227e0d9aea57577382
Run description: This run uses our own model based on the Apache Lucene search engine. In this run, all the patient profiles and the condition, inclusion criteria and exclusion criteria for each clinical study were converted to UMLS Concept Unique Identifiers (CUIs). While each patient profile was considered as a query, all the clinical study condition and inclusion criteria CUIs were organized into a database. For each patient, the search results were sorted using the Lucene scores searched against the database.

CuiIncExc¶

Run ID: CuiIncExc
Participant: GU_BioMed
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 38da8ae1f18a9485bf9b882bf4f56b43
Run description: This run uses our own model based on the Apache Lucene search engine. In this run, all the patient profiles and the condition, inclusion criteria and exclusion criteria for each clinical study were converted to UMLS Concept Unique Identifiers (CUIs). While all the clinical study condition and inclusion criteria CUIs were put in one database, the exclusion criteria CUIs were organized into another database. Using each patient profile with CUIs as a query, the search results were sorted using the difference of the two Lucene scores searched against the two databases, respectively.

damoebr¶

Run ID: damoebr
Participant: ALIBABA
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: a84545ca4882b4e3a8046f73f0866aec
Run description: We use our own embedding-based first-stage retrieval system.

damoebrsigir¶

Run ID: damoebrsigir
Participant: ALIBABA
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: 02c4e87a634ee77190739b6301be3b45
Run description: We use our own embedding-based first-stage retrieval system and a reranker pre-trained by the SIGIR dataset.

damoebrtog¶

Run ID: damoebrtog
Participant: ALIBABA
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: fc33eed95bc0ec8f11197c0d2b4521fe
Run description: We use our own embedding-based first-stage retrieval system.

damohitl¶

Run ID: damohitl
Participant: ALIBABA
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: manual
Task: primary
MD5: 20e094984287bdd21fa6de68fcae77f8
Run description: We use our own embedding-based first-stage retrieval system and an active learning-based reranker.

damohitls¶

Run ID: damohitls
Participant: ALIBABA
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: manual
Task: primary
MD5: 02bac216e6d22356c730d3413eca824c
Run description: We use our own embedding-based first-stage retrieval system and an active learning-based reranker.

desc_rm3¶

Results | Participants | Input | Summary | Appendix

Run ID: desc_rm3
Participant: h2oloo
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 402c9312c3b40e17f3366e29cd298956
Run description: Pyserini's RM3 on the topic description. Default parameters.

elastic¶

Results | Participants | Input | Summary | Appendix

Run ID: elastic
Participant: NTU_NLP
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: cee9561b22abdbd95030a8687e3cbc80
Run description: elasticsearch-based model and search by Amazon Comprehend Medical parsed weight

embed_dense¶

Results | Participants | Input | Summary | Appendix

Run ID: embed_dense
Participant: HEG_Geneva
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: d138e844008fd88dafb6b19b2a5bf65d
Run description: RRF combination of two transformer-based models: One sentence embedding and one re-ranking transformer trained on MS-Marco

f_0_mt5¶

Results | Participants | Input | Summary | Appendix

Run ID: f_0_mt5
Participant: h2oloo
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: bbbfc947f4bbf12261c69001b7af34c4
Run description: First: RRF of 41 different runs, 40 of which run Pyserini's RM3 on queries generated by dT5q and 1 using the standard topic description. Default parameters. Second: 0-shot med-monoT5 3b reranks all 6-length, 3-stride segments using the title, condition and eligibility fields

f_d2q_rm3¶

Results | Participants | Input | Summary | Appendix

Run ID: f_d2q_rm3
Participant: h2oloo
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: a270c79b472c471a29be676076199a76
Run description: RRF of 41 different runs, 40 of which run Pyserini's RM3 on queries generated by dT5q and 1 using the standard topic description. Default parameters.

f_t_mt5¶

Results | Participants | Input | Summary | Appendix

Run ID: f_t_mt5
Participant: h2oloo
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 7fdf13f3b2f63e871f86bb3c507dbf92
Run description: First: RRF of 41 different runs, 40 of which run Pyserini's RM3 on queries generated by dT5q and 1 using the standard topic description. Default parameters. Second: Trained med-monoT5 3b (on Koopman & Zuccon (SIGIR 2016)) reranks all 6-length, 3-stride segments using the title, condition and eligibility fields

f_t_mt5_2¶

Results | Participants | Input | Summary | Appendix

Run ID: f_t_mt5_2
Participant: h2oloo
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 2556b470bf9562989b98e84850233f68
Run description: First: RRF of 41 different runs, 40 of which run Pyserini's RM3 on queries generated by dT5q and 1 using the standard topic description. Default parameters. Second: Trained med-monoT5 3b (on Koopman & Zuccon (SIGIR 2016)) reranks all 6-length, 3-stride segments to select best eligibility segment and best description segment which are combined and finally reranked.

fifth_run¶

Run ID: fifth_run
Participant: IRUniDUE
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 32b725b0a812e13ca9a134775cef8b42
Run description: BM25 for initial ranking on the clinical trial detailed description and eligibility inclusion criteria with gender and age filtering. Then re-rank the documents by calculating a membership function between the original BM25 scores and bm25 scores on the extracted entities using "en_core_sci_scibert" from the topics and eligibility criteria.

first_run¶

Run ID: first_run
Participant: IRUniDUE
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 19ff1ec0302eb0cfd680516953cb621c
Run description: BM25 for initial ranking on the eligibility inclusion criteria with gender and age filtering, and then calculate negative scores from the eligibility exclusion criteria as a membership function.

fourth_run¶

Run ID: fourth_run
Participant: IRUniDUE
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: b619a9ab1b69398b814a7e7b041f487a
Run description: BM25 for initial ranking on the clinical trial detailed description and eligibility inclusion criteria with gender and age filtering. Then re-rank the documents by calculating the cousin similarity between the topic Bio_ClinicalBERT and the clinical trial detailed description.

FUH_T0¶

Results | Participants | Input | Summary | Appendix

Run ID: FUH_T0
Participant: FUH
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: 0e1e8638dedac601562937cfc4dc331d
Run description: We used our own BM25 implementation with default configuration for ranking. We used MeSH Terms and HPO for indexation. Extraction of terms was done without a confidence threshold.

FUH_T30¶

Results | Participants | Input | Summary | Appendix

Run ID: FUH_T30
Participant: FUH
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: b82b877245a21ca5dfac671d91ee985c
Run description: We used our own BM25 implementation with default configuration for ranking. We used MeSH Terms and HPO for indexation. Extraction of terms was done with a confidence threshold of 30%.

FUHT0NoHpo¶

Results | Participants | Input | Summary | Appendix

Run ID: FUHT0NoHpo
Participant: FUH
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: 3b76007b04e9755fa3f98f24696ebdaf
Run description: We used our own BM25 implementation with default configuration for ranking. We used MeSH Terms. Extraction of terms was done without a confidence threshold.

FUHT30NoHpo¶

Results | Participants | Input | Summary | Appendix

Run ID: FUHT30NoHpo
Participant: FUH
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: 33830c0d492e1260cd5b71c0887a30d9
Run description: We used our own BM25 implementation with default configuration for ranking. We used MeSH Terms. Extraction of terms was done without a confidence threshold.

FUHTD30TQ0¶

Results | Participants | Input | Summary | Appendix

Run ID: FUHTD30TQ0
Participant: FUH
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: 2c430a601aa2e28de65b85c8e8ad8666
Run description: We used our own BM25 implementation with default configuration for ranking. We used MeSH Terms and HPO. Extraction of terms from Documents was made with 30% confidence threshold and Queries with 0% confidence threshold.

IBMAUTOGT¶

Run ID: IBMAUTOGT
Participant: IBMResearch
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: f11f735d8690209f7748625d8d1a4705
Run description: This system consists of three modules: 1) query generation module 2) retrieval module and 3) neural re-ranker. The query generation module that relied on IBM Watson Annotator for Clinical Data service, a medical domain NLP service featuring a variety of annotators for detecting metadata such as entities, concepts, concept values, negated spans, hypothetical spans, and a collection of annotators that detect, normalize, and code medical and social findings from unstructured clinical data. Next, based on the observations we made through experimentation on SIGIR-2016 corpora, we developed several heuristics to assign weights to the extracted metadata that can serve as weighted adhoc queries. Finally, these weights along with the metadata, are fed into the retrieval module that includes a) Lucene and b) transformer based semantic textual similarity (STS) model. Next, we select top-N clinical trials for each topic using the ensemble of scores from the retrieval model that are fed into a neural re-ranker. The neural re-ranker is a novel deep learning pipeline that includes two deep learning architectures. As a first step, we trained a state-of-the-art joint concept and relation extraction model using Chia, a large-annotated corpus of clinical trial eligibility criteria for concept and relation extraction from topics. In the next step, we generated silver standard training data by jointly leveraging the structured and unstructured information in MIMIC-III clinical database and structured information in clinical trials. This results approximately 700K potential positive candidates which we used to train a transformer-based learning-to-rank model that leverages outputs from CHIA in addition to novel attention mechanisms to re-rank the relevant clinical trials given a topic.

IBMLucene¶

Run ID: IBMLucene
Participant: IBMResearch
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 951c81f35df8ac875c2e8a9c8dfb8ce6
Run description: This system consists of two modules 1) query generation module and 2) retriever. As a first step, we developed an unsupervised query generation module that relied on IBM Watson Annotator for Clinical Data service, a medical domain NLP service featuring a variety of annotators for detecting metadata such as entities, concepts, concept values, negated spans, hypothetical spans, and a collection of annotators that detect, normalize, and code medical and social findings from unstructured clinical data. Next, based on the observations we made through experimentation on SIGIR-2016 corpora, we developed several heuristics to assign weights to the extracted metadata that can serve as weighted adhoc queries. Finally, these weights along with the metadata, are used to retrieve trials using a lucene index. Trials are indexed with the condition and intervention name. We search these fields using the weighted ad-hoc queries, with the weights serving as boosts, and rank by BM25 score.

IBMSIGIR¶

Run ID: IBMSIGIR
Participant: IBMResearch
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 5882628595567fb6a4bd33f87e0ba368
Run description: This system consists of three modules: 1) query generation module 2) retrieval module and 3) neural re-ranker. The query generation module that relied on IBM Watson Annotator for Clinical Data service, a medical domain NLP service featuring a variety of annotators for detecting metadata such as entities, concepts, concept values, negated spans, hypothetical spans, and a collection of annotators that detect, normalize, and code medical and social findings from unstructured clinical data. Next, based on the observations we made through experimentation on SIGIR-2016 corpora, we developed several heuristics to assign weights to the extracted metadata that can serve as weighted adhoc queries. Finally, these weights along with the metadata, are fed into the retrieval module that includes a) Lucene and b) transformer based semantic textual similarity (STS) model. Next, we select top-N clinical trials for each topic using the ensemble of scores from the retrieval model that are fed into a neural re-ranker. The neural re-ranker is a novel deep learning pipeline that includes two deep learning architectures. As a first step, we trained a state-of-the-art joint concept and relation extraction model using Chia, a large-annotated corpus of clinical trial eligibility criteria for concept and relation extraction from topics. In the next step, we use SIGIR-2016 corpora to train a transformer-based learning-to-rank model that leverages outputs from CHIA in addition to novel attention mechanisms to re-rank the relevant clinical trials given a topic.

IBMSIGIRACT¶

Run ID: IBMSIGIRACT
Participant: IBMResearch
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 9d8784c9d0cde97871ecac7c33448fc3
Run description: This system consists of three modules: 1) query generation module 2) retrieval module and 3) neural re-ranker. The query generation module that relied on IBM Watson Annotator for Clinical Data service, a medical domain NLP service featuring a variety of annotators for detecting metadata such as entities, concepts, concept values, negated spans, hypothetical spans, and a collection of annotators that detect, normalize, and code medical and social findings from unstructured clinical data. Next, based on the observations we made through experimentation on SIGIR-2016 corpora, we developed several heuristics to assign weights to the extracted metadata that can serve as weighted adhoc queries. Finally, these weights along with the metadata, are fed into the retrieval module that includes a) Lucene and b) transformer based semantic textual similarity (STS) model. Next, we select top-N clinical trials for each topic using the ensemble of scores from the retrieval model that are fed into a neural re-ranker. The neural re-ranker is a novel deep learning pipeline that includes two deep learning architectures. As a first step, we trained state-of-the-art joint concept and relation extraction model using Chia, a large-annotated corpus of clinical trial eligibility criteria for concept and relation extraction from topics. In the next step, we use SIGIR-2016 corpora to train a transformer-based learning-to-rank model that leverages outputs from CHIA in addition to novel attention mechanisms to re-rank the relevant clinical trials given a topic. We believe it is worthwhile to understand the true impact of our system as a real-world use case, thus we match the patients against only the trials that are Currently Active (Recruiting, Not-yet-recruiting, Enrolling by invitation, Active not recruiting).

IBMSTS¶

Run ID: IBMSTS
Participant: IBMResearch
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 050c268851200efa558ead19042e927d
Run description: This system consists of two modules 1) query generation module and 2) neural retriever. As a first step, we developed an unsupervised query generation module that relied on IBM Watson Annotator for Clinical Data service, a medical domain NLP service featuring a variety of annotators for detecting metadata such as entities, concepts, concept values, negated spans, hypothetical spans, and a collection of annotators that detect, normalize, and code medical and social findings from unstructured clinical data. Next, based on the observations we made through experimentation on SIGIR-2016 corpora, we developed several heuristics to assign weights to the extracted metadata that can serve as weighted adhoc queries. Finally, these weights along with the metadata, are used to retrieve trials using a transformer based semantic textual similarity (STS) model. In summary, Low-dimensional representations are obtained for all the conditions and interventions present in the trials and the extracted metadata of each topic. Next, we measure weight normalized pairwise cosine similarity between a topic and the low-dimensional representations of all the trails to rank them.

ielabr1¶

Results | Participants | Input | Summary | Appendix

Run ID: ielabr1
Participant: ielab
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: bd709a7212b5ee200951ff272faab9e0
Run description: pyterrier, bm25 default

ielabr2¶

Results | Participants | Input | Summary | Appendix

Run ID: ielabr2
Participant: ielab
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: fb2586b1f25175871a6bb10271001f0d
Run description: pyterrier, bm25 default, BioBERT

ielabr3¶

Results | Participants | Input | Summary | Appendix

Run ID: ielabr3
Participant: ielab
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 71eadbb48d70a697362734e0f9ea1108
Run description: pyterrier, bm25 default, Dirichlet default, PL default

ielabr4¶

Results | Participants | Input | Summary | Appendix

Run ID: ielabr4
Participant: ielab
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: manual
Task: primary
MD5: 67ef4f005aaae2e04fef40f6c8e4e06a
Run description: pyterrier, bm25 default

IKR3_BSL¶

Run ID: IKR3_BSL
Participant: UNIMIB
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 7136b8e93b6385d8609fde05e4be9f3c
Run description: This model implements a keyword extraction method for every topic and then ranks using BM25 and Pyterrier.

IKR3_TT_MW_d¶

Run ID: IKR3_TT_MW_d
Participant: UNIMIB
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 340dc00a8469469f82aee598a04df2e6
Run description: Another Decision theoretic approach in which the whole topic has been used. Topical Relevance has been determined using BM25.

IKR3_TT_MW_k¶

Run ID: IKR3_TT_MW_k
Participant: UNIMIB
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 24191431cf9c7c7c1d15253394207de6
Run description: This is a decision theoretic approach. It employs the TOPSIS model to aggregate the topicality scores of different fields. In this case, the topical scores were calculated using BM25.

imsFused1¶

Run ID: imsFused1
Participant: ims_unipd
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 1031c7cce501c11b1f683e124443185f
Run description: Fusion of BART RM3 + T5 RM3

imsFused2¶

Run ID: imsFused2
Participant: ims_unipd
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 0cc561e7b9f966ad3dbe65933ebced75
Run description: Fusion of all the previous runs

ittc1¶

Run ID: ittc1
Participant: ITTC-AIMedTech
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: dbabf61f146e05cf7eb1840507659577
Run description: BM25 with negated entities (query and document); on the document side we reversed the negated/affirmed entities in exclusions and combined them in the same list with inclusions

ittc2¶

Run ID: ittc2
Participant: ITTC-AIMedTech
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 9fa351b0c1450a16ac1f2d1254aec302
Run description: BM25 plus the most comprehensive query processing (with negation and metadata such as age), matched against the inclusions text, combined keywords and metadata, too; exclusions here are represented in form of reverse negated entities and metadata/clinical variables

ittc3¶

Run ID: ittc3
Participant: ITTC-AIMedTech
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 392e24f5fc1b2d152c26522200fa085e
Run description: BM25 plus matches a the full text of the query against two indexes - one indexing the title and inclusion criteria, the other indexing the exclusion criteria; then the set difference is taken

ittc4¶

Run ID: ittc4
Participant: ITTC-AIMedTech
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: eca8218c87ac60947e6c273be388ddce
Run description: BM25 plus matches keyterms of the query against two indexes - one indexing the title and inclusion criteria, the other indexing the exclusion criteria; then the set difference is taken

ittc5¶

Run ID: ittc5
Participant: ITTC-AIMedTech
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 2b13558e508f7328245f982a79579caf
Run description: BM25 plus matches the key terms of the query against two indexes - one indexing the title and inclusion criteria, the other indexing the exclusion criteria; then the set difference is taken. All negated entities excluded from the query.

jbnu1¶

Results | Participants | Input | Summary | Appendix

Run ID: jbnu1
Participant: jbnu
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: cf1f491e144df9cb04c684b05fb59a94
Run description: query expansion using ICD-10 and Wikipedia

jbnu2¶

Results | Participants | Input | Summary | Appendix

Run ID: jbnu2
Participant: jbnu
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 7f1d0e2bb65e5351728a7131967916ef
Run description: pseudo relevance feedback + query expansion

LaBSE¶

Run ID: LaBSE
Participant: uni_pais_vasco
Track: Clinical Trials
Year: 2021
Submission: 8/11/2021
Type: automatic
Task: primary
MD5: 063338547199451941eee2931552627e
Run description: This model first summarizes the text by extracting medical entities with an NER model trained on I2B2 2010. Next, sentence embeddings are created for these summarized texts using the Language-agnostic BERT sentence embedding model available from Hugging Face. Similarity between documents and topics and computed via cosine similarity.

LaBSE_rerank¶

Run ID: LaBSE_rerank
Participant: uni_pais_vasco
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: 2b4770d340eab8a5ed1ad7385693a532
Run description: This model first uses an NER model trained on I2B2 2010 to extract medical entities from texts. Next, the LaBSE model from sentence-transformers (Hugging Face) is used to create document embedding. Lastly, a reranking is performed by performing a weighted average of the aforementioned sentence embeddings and binary vectors (word present in target text vocabulary).

mix5E10K128¶

Results | Participants | Input | Summary | Appendix

Run ID: mix5E10K128
Participant: NTU_NLP
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: d3a4ddae3253b560af61895838edf2dc
Run description: Siamese BERT-Large model fused with elastic search. Sliding Window overlap is 128, number of elastic search for fused weight sorting is 10000, fused ratio is 0.5

mix6E15K128¶

Results | Participants | Input | Summary | Appendix

Run ID: mix6E15K128
Participant: NTU_NLP
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: fa4b82add004cf38124489c25c371b50
Run description: Siamese BERT-Large model fused with elastic search. Sliding Window overlap is 128, number of elastic search for fused weight sorting is 15000, fused ratio is 0.6

mix6E5k128¶

Results | Participants | Input | Summary | Appendix

Run ID: mix6E5k128
Participant: NTU_NLP
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 9b5964359382c82e6db0c7d71b7e54b6
Run description: Siamese BERT-Large model fused with elastic search. Sliding Window overlap is 128, number of elastic search for fused weight sorting is 5000, fused ratio is 0.6

mix6E5k64¶

Results | Participants | Input | Summary | Appendix

Run ID: mix6E5k64
Participant: NTU_NLP
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: e91690a008ea0048d308e30d0fda8afa
Run description: Siamese BERT-Large model fused with elastic search. Sliding Window overlap is 64, number of elastic search for fused weight sorting is 5000, fused ratio is 0.6

postproc¶

Run ID: postproc
Participant: DOSSIER
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 6fb92902e82ca83f52e6682ae599a34b
Run description: bm25 + postprocessing

pozAbbrMesh¶

Run ID: pozAbbrMesh
Participant: POZNAN
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 31719296c5801b86fbe290c4234b0750
Run description: elasticsearch, BM25, fields: summary, description, titles (both), intervention, mesh terms, condition

pozAddTerms¶

Run ID: pozAddTerms
Participant: POZNAN
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: manual
Task: primary
MD5: 7178d0007532ed947a0e1dd3a51226b5
Run description: elasticsearch, BM25, fields: summary, description, titles (both), intervention, mesh terms, condition

pozFulltext¶

Run ID: pozFulltext
Participant: POZNAN
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: c05123d61c650c53f42fcb4bcb7fcc74
Run description: elasticsearch, BM25, fields: summary, description, titles (both)

pozMeshTerms¶

Run ID: pozMeshTerms
Participant: POZNAN
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 86967349f8e5d96a3b81c936c827206c
Run description: elasticsearch, BM25, fields: summary, description, titles (both), condition, intervention, mesh terms

pozTermFds¶

Run ID: pozTermFds
Participant: POZNAN
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 1d713cd51bbb58aab470a8d175f776a8
Run description: elasticsearch, BM25, fields: condition, intervention, mesh terms

rerank2000¶

Run ID: rerank2000
Participant: DOSSIER
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 7306f487e5197bc1afe298f4f2769aba
Run description: bm25 + postprocessing + reranking based on eligibility with pretrained allenai-specter model

RM3Filtered¶

Run ID: RM3Filtered
Participant: ims_unipd
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: da28358e7424fff7e0384d115293c953
Run description: BM25 + RM3 pseudo relevance feedback

rrf_all¶

Results | Participants | Input | Summary | Appendix

Run ID: rrf_all
Participant: HEG_Geneva
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: 1c7df6f312e1dcb3d19b12989d78c298
Run description: RRF combination of the 2 BM25 models, the 2 unsupervised transformers and the supervised model fine-tuned on 2015 qrels

RRMATCH¶

Results | Participants | Input | Summary | Appendix

Run ID: RRMATCH
Participant: NISTRR
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 629b49c0ecd0e2564ef16873b6585322
Run description: This uses our own Parmenides structural phrase-matching retrieval model.

RUN0¶

Run ID: RUN0
Participant: clinical_trials
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: df5589cd8cb4d687f237d50152463c8a
Run description: Our own model based on open source algorithms. 1. "LIKE" query to the database with named entities extracted from topics. We trained named entities extraction on the CHIA corpus. 2. Extracting named entities from trials found. 3. Further filtering on exclusion criteria. 4. Sorting based on cosine distances.

RUN1FREQS¶

Run ID: RUN1FREQS
Participant: clinical_trials
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 078795a3a55f1eadf6e5c39974f03c6e
Run description: Our own model based on open source algorithms. 1."LIKE" queries to the trials database by named entities extracted from topics. 2. Trials are sorted by named entities frequencies (rare diseases are of highest relevance) 3. Filtering by named entities extracted from exclusion criteria.

RUN3SENTS¶

Run ID: RUN3SENTS
Participant: clinical_trials
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: fb18c6af53f8c42c1091e289a61d4e9a
Run description: Our own based on open-source algorithms. Everything was as in our first run, but we used sentence embeddings instead of word embeddings.

seansct01¶

Results | Participants | Input | Summary | Appendix

Run ID: seansct01
Participant: V-Ryerson
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: cd0f017d1b43da5f802c77fbb478e254
Run description: BM25+

second_run¶

Run ID: second_run
Participant: IRUniDUE
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: ae0a86dfdb7d716596888329bfa61382
Run description: BM25 for initial ranking on the clinical trial detailed description with gender and age filtering. Then re-rank the documents by calculating the BM25 for both eligibility exclusion and inclusion criteria. The final score is the subtraction of the inclusion and exclusion scores.

SIBTMct1¶

Run ID: SIBTMct1
Participant: BITEM
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 9ca2c44fd7c4529c171add6d98a993ac
Run description: Upstream from the queries: Topics and CT processing: annotations based on NCIt, ICD, SNOMED and MESH ontologies. Common pipeline for each run (query): - Gender and age of patient have to be in accordance to the recruitment criteria of the CT (exact gender or "all"; age min <= age of patient <= age max) - annotations codes in some CT fields (brief title, official title, brief summary, detailed description, condition, study population, criteria, inclusion criteria, keywords and mesh terms) - If annotation code of topic found in the "exclusion criteria" of the CT, CT is rejected. Query details: MUST: Annotations codes ICD are required in particular CT fields. SHOULD: Annotations codes NCI, SNOMED and MESH are required in particular CT fields. Large boost for matching in "inclusion criteria" field. Boost for NCI and MESH matching. If no results for topic, add results from run 2, a one grade more relaxed query.

SIBTMct2¶

Run ID: SIBTMct2
Participant: BITEM
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: c9b2e8bf668a333e80a7332639ef8649
Run description: Upstream from the queries: Topics and CT processing: annotations based on NCIt, ICD, SNOMED and MESH ontologies. Common pipeline for each run (query): - Gender and age of patient have to be in accordance to the recruitment criteria of the CT (exact gender or "all"; age min <= age of patient <= age max) - annotations codes in some CT fields (brief title, official title, brief summary, detailed description, condition, study population, criteria, inclusion criteria, keywords and mesh terms) - If annotation code of topic found in the "exclusion criteria" of the CT, CT is rejected. Query details: SHOULD: All annotations codes NCI, SNOMED and MESH should be present in particular CT fields. Large boost for matching in "inclusion criteria" field. Boost for NCI, ICD and MESH matching.

SIBTMct3¶

Run ID: SIBTMct3
Participant: BITEM
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 0d96a114b8bf952d04e1c9e6d0e83882
Run description: Upstream from the queries: Topics and CT processing: annotations based on NCIt, ICD, SNOMED and MESH ontologies. Common pipeline for each run (query): - Gender and age of patient have to be in accordance to the recruitment criteria of the CT (exact gender or "all"; age min <= age of patient <= age max) - annotations codes in some CT fields (brief title, official title, brief summary, detailed description, condition, study population, criteria, inclusion criteria, keywords and mesh terms) - If annotation code of topic found in the "exclusion criteria" of the CT, CT is rejected. Query details: SHOULD: All annotations codes NCI, SNOMED and MESH should be present in particular CT fields. Large boost for matching in "inclusion criteria" field. Boost for matching in "keywords" field. Boost for NCI, ICD and MESH matching.

SIBTMct4¶

Run ID: SIBTMct4
Participant: BITEM
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 63110a914cec48af3142fd367c267a56
Run description: Upstream from the queries: Topics and CT processing: annotations based on NCIt, ICD, SNOMED and MESH ontologies. Common pipeline for each run (query): - Gender and age of patient have to be in accordance to the recruitment criteria of the CT (exact gender or "all"; age min <= age of patient <= age max) - annotations codes in some CT fields (brief title, official title, brief summary, detailed description, condition, study population, criteria, inclusion criteria, keywords and mesh terms) - If annotation code of topic found in the "exclusion criteria" of the CT, CT is rejected. Query details: SHOULD: All annotations codes NCI, SNOMED and MESH should be present in particular CT fields. Large boost for matching in "inclusion criteria" field. Boost for matching in "mesh_terms" field. Boost for NCI, ICD and MESH matching.

SIBTMct5¶

Run ID: SIBTMct5
Participant: BITEM
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 04dabe416e187b71ef60383f80056945
Run description: Upstream from the queries: Topics and CT processing: annotations based on NCIt, ICD, SNOMED and MESH ontologies. Common pipeline for each run (query): - Gender and age of patient have to be in accordance to the recruitment criteria of the CT (exact gender or "all"; age min <= age of patient <= age max) - annotations codes in some CT fields (brief title, official title, brief summary, detailed description, condition, study population, criteria, inclusion criteria, keywords and mesh terms) - If annotation code of topic found in the "exclusion criteria" of the CT, CT is rejected. Query details: SHOULD: All annotations codes NCI, SNOMED and MESH should be present in particular CT fields. Large boost for matching in "inclusion criteria" field. Boost for matching in "description_detailed" field. Boost for NCI, ICD and MESH matching.

spec_rrk_fqv¶

Run ID: spec_rrk_fqv
Participant: uni_pais_vasco
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 89408acd8da66e343716cde0a4be14d5
Run description: This model first uses an NER model trained on I2B2 2010 to extract medical entities from texts. Next, the allenai-specter model from sentence-transformers (Hugging Face) is used to create document embedding. Lastly, a reranking is performed by performing a weighted average of the aforementioned sentence embeddings and frequency vectors (frequency of words present in target text vocabulary).

spect_rerank¶

Run ID: spect_rerank
Participant: uni_pais_vasco
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: 10abcc482262038b5cfcdc8c00adf0d8
Run description: This model first uses an NER model trained on I2B2 2010 to extract medical entities from texts. Next, the allenai-specter model from sentence-transformers (Hugging Face) is used to create document embedding. Lastly, a reranking is performed by performing a weighted average of the aforementioned sentence embeddings and binary vectors (word present in target text vocabulary).

specter¶

Run ID: specter
Participant: uni_pais_vasco
Track: Clinical Trials
Year: 2021
Submission: 8/11/2021
Type: automatic
Task: primary
MD5: 4ed09cad5ebb5737e055d868cc41b399
Run description: This model first summarizes the text by extracting medical entities with an NER model trained on I2B2 2010. Next, sentence embeddings are created for these summarized texts using the allenai-specter sentence transformer model available from Hugging Face. Similarity between documents and topics and computed via cosine similarity.

superv_map¶

Results | Participants | Input | Summary | Appendix

Run ID: superv_map
Participant: HEG_Geneva
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: 3193a49d5beb932101044a139dd6cbdf
Run description: RRF combination of BM25 on metamap-enriched queries, and sentence embedding with an MLP fine-tuned on the 2015 qrels.

T5RM3Filt¶

Run ID: T5RM3Filt
Participant: ims_unipd
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 97a5d14fc54c608256ed82c402a1a301
Run description: BM25 + T5 Summary + RM3 pseudo RF

tdminerrun1¶

Run ID: tdminerrun1
Participant: TDMINER
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: manual
Task: primary
MD5: b75a96e03bbb7f655ebbc2aa06afc39f
Run description: Own model. IE-based method.

tdminerrun2¶

Run ID: tdminerrun2
Participant: TDMINER
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: manual
Task: primary
MD5: 1d1ceb7d38401822490feac554e2b85d
Run description: Own model. IE-based method.

tdminerrun3¶

Run ID: tdminerrun3
Participant: TDMINER
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: manual
Task: primary
MD5: 417bb364bf7a9ed1ca9ffb46ad1b2e86
Run description: Own model. IE-based method.

tdminerrun4¶

Run ID: tdminerrun4
Participant: TDMINER
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: manual
Task: primary
MD5: c54ad68a8276ff27936b7644b1c14ad4
Run description: Own model. IE-based method.

third_run¶

Run ID: third_run
Participant: IRUniDUE
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 3f19f962ead51075f81fcc5c661d56a8
Run description: BM25 for ranking documents based on the clinical trial eligibility inclusion and the boosted topic entities extracted using "en_core_sci_scibert" on the detailed description with gender and age filtering.

TxtInc¶

Run ID: TxtInc
Participant: GU_BioMed
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 43697b24c4c07f9d78466d8020a0be69
Run description: This run uses our own model based on free-text search using the Apache Lucene search engine. While each patient profile was considered as a query, all the clinical study condition and inclusion criteria free texts were organized into a database. For each patient, the search results were sorted using the Lucene scores searched against the database.

TxtIncExc¶

Run ID: TxtIncExc
Participant: GU_BioMed
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 1f74f7df775518681c8d997c9f8344d3
Run description: This run uses our own model based on free-text search using the Apache Lucene search engine. While all the clinical study condition and inclusion criteria were put in one database, the exclusion criteria were organized into another database. Using each patient profile as a query, the search results were sorted using the difference of the two Lucene scores searched against the two databases, respectively.

TxtIncExcExp¶

Run ID: TxtIncExcExp
Participant: GU_BioMed
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 22418cf9f59eb73b9b8c7f6e8700594b
Run description: This run uses our own model based on free-text search using the Apache Lucene search engine. In this run, all the condition terms for each clinical study were expanded with their child terms in MeSH. While all the expanded condition and the inclusion criteria were put in one database, the exclusion criteria were organized into another database. Using each patient profile as a query, the search results were sorted using the difference of the two Lucene scores searched against the two databases, respectively.

UNM_4¶

Run ID: UNM_4
Participant: UNIMIB
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: 732b42371b299c56e91a44ff5fd4d28f
Run description: DFR from pyterrier and a Bert model fine tuned for reranking

UNM_5¶

Run ID: UNM_5
Participant: UNIMIB
Track: Clinical Trials
Year: 2021
Submission: 8/17/2021
Type: automatic
Task: primary
MD5: d0aab4e06a0b2bc1d8f59250841ce02a
Run description: DFR from pyterrier and a Bert model fine tuned for reranking

UNTIIARUN1¶

Run ID: UNTIIARUN1
Participant: UNTIIA
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: ae8d3ea25c3282c2b176aa7ec8fed2c5
Run description: BM25 from Elastic search (https://www.elastic.co/)

UNTIIARUN2¶

Run ID: UNTIIARUN2
Participant: UNTIIA
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: manual
Task: primary
MD5: 2bb623c47be95965b3d2c4330d626a84
Run description: BM25 from Elastic search (https://www.elastic.co/)

UNTIIARUN3¶

Run ID: UNTIIARUN3
Participant: UNTIIA
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: manual
Task: primary
MD5: d08425f1576faefdfcf7fa392a07040f
Run description: BM25 from Elastic search (https://www.elastic.co/)

UNTIIARUN4¶

Run ID: UNTIIARUN4
Participant: UNTIIA
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: manual
Task: primary
MD5: 69d6b4ad81ac0b4832455e5dfac3a5d4
Run description: BM25 from Elastic search (https://www.elastic.co/)

UNTIIARUN5¶

Run ID: UNTIIARUN5
Participant: UNTIIA
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: manual
Task: primary
MD5: f0937ed602b5358c58fc2f01e8ed5465
Run description: BM25 from Elastic search (https://www.elastic.co/)

vohbm25¶

Results | Participants | Input | Summary | Appendix

Run ID: vohbm25
Participant: vohcolab
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: f62013bb2b6b741dc002e10c80e93093
Run description: BM25 baseline tuned with CSIRO collection

vohbm25KE¶

Results | Participants | Input | Summary | Appendix

Run ID: vohbm25KE
Participant: vohcolab
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 2c880c83580c26d657a82dc87981cfcc
Run description: bm25 with keyword extraction from query

vohCBsim¶

Results | Participants | Input | Summary | Appendix

Run ID: vohCBsim
Participant: vohcolab
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: ab1310012b585da865cfef4cb23ee2c4
Run description: bm25 with clinical bert embedding cosine similarity vs topics reranking

vohDemo¶

Results | Participants | Input | Summary | Appendix

Run ID: vohDemo
Participant: vohcolab
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 968c558735a9ba491d3319e42ff1c232
Run description: bm25 to retrieve 10k, linear re-ranking with gender and age features

vohl2r5¶

Results | Participants | Input | Summary | Appendix

Run ID: vohl2r5
Participant: vohcolab
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 1c1a7b2955cecd008882e05d0fae371e
Run description: learn 2 rank with bm25, age matching, gender matching and MESH related features.

wpm_biobert¶

Run ID: wpm_biobert
Participant: wispermedtxt
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 7bdbb7014b4e2ab96a9ba5c350433178
Run description: Dense (biobert embeddings) and sparse (tfidf) weighted similarity between topics and trials (title + brief summary). All trials were filtered for including age and gender of each topic. Age and gender were previously extracted from topics using regular expressions.

wpm_bmre¶

Run ID: wpm_bmre
Participant: wispermedtxt
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: b329d0c1548b5bd6127c04f122cbc0df
Run description: BM25 Ranking with Elasticsearch indexing. All results were then reranked based on weighted tfidf & biobert embedding topic to title + brief_summary similarity.

wpm_CBert¶

Run ID: wpm_CBert
Participant: wispermedtxt
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 00896f0713b68ba8a365677584bdcfcb
Run description: Bio_ClinicalBERT Embeddings: Cosine Similarity between Topics and Trials (Title + Brief_Summary). Trials were filtered for matching age + gender (obtained through regular expressions).

wpm_critumls¶

Run ID: wpm_critumls
Participant: wispermedtxt
Track: Clinical Trials
Year: 2021
Submission: 8/19/2021
Type: automatic
Task: primary
MD5: 997afe32981a1ca4ca050736e36163ca
Run description: Trials were lexicographically ordered: First by shared UMLS codes in trial title and topics, second by jaccard index for inclusion criteria and finally by jaccard index for exclusion criteria. The UMLS codes were weighted inversely to the number of trials in which they occur.

wpm_KWBM25¶

Run ID: wpm_KWBM25
Participant: wispermedtxt
Track: Clinical Trials
Year: 2021
Submission: 8/18/2021
Type: automatic
Task: primary
MD5: 520aa446100045a9732ee3233891390a
Run description: Keyword extraction from topics using KeyBert with the model Bio_ClinicalBERT. Ranking based on BM25 between keywords and title + brief_summary. Trials have been filtered to match age and gender.