Runs - Incident Streams 2018¶

cbnuC1¶

Participants | Proceedings | Input | Appendix

Run ID: cbnuC1
Participant: cbnu
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: ca79f0323b1ad3976a5c52dfb0d5142f
Run description: Conceptual representation for Convolutional Neural Networks with Class Activation Map.

cbnuC2¶

Participants | Proceedings | Input | Appendix

Run ID: cbnuC2
Participant: cbnu
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: 3f9743d63930fd5d323b66ccdfb53cb1
Run description: Conceptual representation for Convolutional Neural Networks with Class Activation Map. Twitter frequent user and url info.

cbnuS1¶

Participants | Proceedings | Input | Appendix

Run ID: cbnuS1
Participant: cbnu
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: db3927e609e4782099ae391622e9d178
Run description: Conceptual representation for SVM classification.

cbnuS2¶

Participants | Proceedings | Input | Appendix

Run ID: cbnuS2
Participant: cbnu
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: 42e96157b3641ce4f14c6ab5002acfb7
Run description: Conceptual representation for SVM classification. Twitter frequent user and url info.

DLR_Augmented¶

Participants | Input | Appendix

Run ID: DLR_Augmented
Participant: DLR_DW_BWS
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: dd89284d5c8d3d723ebf7f961a3e7013
Run description: CNN trained on TREC training data augmented with random back-and-forth translations. External resources: To overcome issues caused by missing, sparse and imbalanced training data, additional training tweets were added manually to the initial data set. The final training data set contains at least ~50 training tweets per class. Source: CrisisLexT26 and EMTerms (http://crisislex.org/crisis-lexicon.html).

DLR_Baseline¶

Participants | Input | Appendix

Run ID: DLR_Baseline
Participant: DLR_DW_BWS
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: c22fe2a0850f79121847d8fa9300ede0
Run description: This is our baseline, a simple logistic regression using token- and character n-grams, tweet sentiment, if the user is verified and the number of retweets, likes and media attachments. We want to use it to compare the effect of the various improvements and ideas put into the more sophisticated models.

DLR_Fusion¶

Participants | Input | Appendix

Run ID: DLR_Fusion
Participant: DLR_DW_BWS
Track: Incident Streams
Year: 2018
Submission: 9/4/2018
Type: auto
Task: main
MD5: d68be77e1da009784dfdfc74f37a080f
Run description: A fusion network of CNNs trained on two other corpora and on the TREC-IS training data augmented with random back-and-forth translations. [corrected submission]

DLR_Simple_CNN¶

Participants | Input | Appendix

Run ID: DLR_Simple_CNN
Participant: DLR_DW_BWS
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: 78af8335a5378e312f8a810019d5ba9e
Run description: A simple CNN with a single convolution layer was used. The input word vectors were obtained by using the CrisisNLP_word2vec_model-v1.0 (CrisisNLP resouce #1, http://crisisnlp.qcri.org/lrec2016/lrec2016.html). External resources: To overcome issues caused by missing, sparse and imbalanced training data, additional training tweets were added manually to the initial data set. The final training data set contains at least ~50 training tweets per class. Source: CrisisLexT26 and EMTerms (http://crisislex.org/crisis-lexicon.html).

IITBHU1¶

Participants | Proceedings | Input | Appendix

Run ID: IITBHU1
Participant: IIT-BHU
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: manual
Task: main
MD5: 0091d9ea8787ab4739544ec1c18bae67
Run description: From the indicators given in the training data, a distribution of indicator term frequencies is obtained (after stemming) for each event for each category. Another term frequency distribution is obtained for each event for each category using the text of the training tweets (after stemming). Queries for each of the 6 types of events are formulated using the above two distributions and the descriptions of some categories given in the user profiles.

IITBHU12¶

Participants | Proceedings | Input | Appendix

Run ID: IITBHU12
Participant: IIT-BHU
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: manual
Task: main
MD5: cadf399b3c4bf256f0e9bf0e26959516
Run description: From the indicators given in the training data, a distribution of indicator term frequencies is obtained (after stemming) for each event for each category. Another term frequency distribution is obtained for each event for each category using the text of the training tweets (after stemming). Queries for each of the 6 types of events are formulated using the above two distributions and the descriptions of some categories given in the user profiles.

KDEIS1_CLSTM¶

Participants | Proceedings | Input | Appendix

Run ID: KDEIS1_CLSTM
Participant: KDEIS
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: da65bf122c5478a06f94b12b9e18815e
Run description: For each test events, we considered the data as a single batch. We then identified the non-English tweets and classified them as irrelevant. Next, we design a rule-based classifier by exploiting the indicator terms of the given training set to classify the tweets. For the tweets that are unclassified by the rule-based classifier, we consider the combined prediction score from the convolutional long short-term memory (C-LSTM) based deep learning architecture and SVM classifier with a rich set of hand-crafted features to classify these tweets.

KDEIS2_ACBLSTM¶

Participants | Proceedings | Input | Appendix

Run ID: KDEIS2_ACBLSTM
Participant: KDEIS
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: 3dae310efa2960407c088be3b3fa48a0
Run description: For each test events, we considered the data as a single batch. We then identified the non-English tweets and classified them as irrelevant. Next, we design a rule-based classifier by exploiting the indicator terms of the given training set to classify the tweets. For the tweets that are unclassified by the rule-based classifier, we consider the combined prediction score from the attention based convolutional bi-directional long short-term memory (C-BLSTM) architecture and SVM classifier with a rich set of hand-crafted features to classify these tweets.

KDEIS3_ACSBLSTM¶

Participants | Proceedings | Input | Appendix

Run ID: KDEIS3_ACSBLSTM
Participant: KDEIS
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: b2bfab77e8e5547bbe2ade3df5fc49aa
Run description: For each test events, we considered the data as a single batch. We then identified the non-English tweets and classified them as irrelevant. Next, we design a rule-based classifier by exploiting the indicator terms of the given training set to classify the tweets. For the tweets that are unclassified by the rule-based classifier, we consider the combined prediction score from the attention based convolutional deep stacked bi-directional long short-term memory (C-SBLSTM) architecture and SVM classifier with a rich set of hand-crafted features to classify these tweets.

KDEIS4_DM¶

Participants | Proceedings | Input | Appendix

Run ID: KDEIS4_DM
Participant: KDEIS
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: 7e5a4a1be34293b7fe2cb4a9d17be5c3
Run description: For each test events, we considered the data as a single batch. We then identified the non-English tweets and classified them as irrelevant. Next, we design a rule-based classifier by exploiting the indicator terms of the given training set to classify the tweets. For the tweets that are unclassified by the rule-based classifier, we consider the combined prediction score from the DeepMoji architecture and SVM classifier with a rich set of hand-crafted features to classify these tweets.

myrun-10¶

Participants | Proceedings | Input | Appendix

Run ID: myrun-10
Participant: EPIC_MR
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: 1a1c4b4ed88b67ec68ebe03bd62472c3
Run description: external resources: Data: Wikipedia libraries: sklearn, nltk, numpy

myrun-11¶

Participants | Proceedings | Input | Appendix

Run ID: myrun-11
Participant: EPIC_MR
Track: Incident Streams
Year: 2018
Submission: 9/2/2018
Type: auto
Task: main
MD5: 74cfb8db4b6d28722df549ae48252181
Run description: External resources Data: Wikipedia Libraries: ntlk, sklearn, numpy

myrun-2¶

Participants | Proceedings | Input | Appendix

Run ID: myrun-2
Participant: EPIC_MR
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: d5d3d7837d3c4004d9cc6ced6fce9645
Run description: external resources: Data: Wikipedia libraries: sklearn, nltk, numpy

myrun-21¶

Participants | Proceedings | Input | Appendix

Run ID: myrun-21
Participant: EPIC_MR
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: f670c32934f08184d7f23d957501e6ec
Run description: external resources Data: Wikipedia libraries: sklearn, nltk, numpy

myrun1¶

Participants | Proceedings | Input | Appendix

Run ID: myrun1
Participant: BJUT
Track: Incident Streams
Year: 2018
Submission: 9/1/2018
Type: auto
Task: main
MD5: 58b0256b043364c1243b0ff1fe74c959
Run description: all data were created by code

myrun2¶

Participants | Proceedings | Input | Appendix

Run ID: myrun2
Participant: BJUT
Track: Incident Streams
Year: 2018
Submission: 9/1/2018
Type: auto
Task: main
MD5: 86525e5010a4283f954771c9816ccaa7
Run description: all data were created by code and manual intervention

NHK_run1¶

Participants | Proceedings | Input | Appendix

Run ID: NHK_run1
Participant: NHK
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: b65e6ba812791e4b6c1f9a815cf087e3
Run description: We used relational-graph convolutional networks (R-GCN) as word embedding, then classify into the information label and its necessity level using feed-forward neural networks. Our method first extracts entities from a input Tweet, then entities are expanded using WordNet into hypernyms, hyponyms, and so on. Our R-GCN considers not only entities appeared in data set, but also those expanded entities, so it can use larger amount of data for models training. As a result, our method can overcome the small training data set scenario, and obtained rather good results in our 10-fold cross validation experiments in macro average of f1 score. We submit the output of the ensemble of 10 models that obtained in each of cross validation.

NHK_run2¶

Participants | Proceedings | Input | Appendix

Run ID: NHK_run2
Participant: NHK
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: 42ca55e8247584bfd8d9ccaa39ada652
Run description: We used Feed Forward Neural Networks (FFNN) with inputting text, date which is created at, and event types, then classify into the information label and its necessity level. As a result, our method can overcome the small training data set scenario, and obtained rather good results in our 10-fold cross validation experiments in macro average of f1 score. We submit the output of the ensemble of 10 models that obtained in each of cross validation.

NHK_run3¶

Participants | Proceedings | Input | Appendix

Run ID: NHK_run3
Participant: NHK
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: 55386a3845abc71d039586d71859fcd7
Run description: We used Feed Forward Neural Networks (FFNN) with inputting text, then classify into the information label and its necessity level As a result, our method can overcome the small training data set scenario, and obtained rather good results in our 10-fold cross validation experiments in micro average of f1 score. We submit the output of the ensemble of 10 models that obtained in each of cross validation.

NHK_run4¶

Participants | Proceedings | Input | Appendix

Run ID: NHK_run4
Participant: NHK
Track: Incident Streams
Year: 2018
Submission: 9/4/2018
Type: auto
Task: main
MD5: a82c4aee58241087dacb0b146123cf9d
Run description: We used Support Vector machines with inputting text, date which is created at, and event types, then classify into the information label and its necessity level. This method is baseline method for us.

SINAI_run1¶

Participants | Proceedings | Input | Appendix

Run ID: SINAI_run1
Participant: SINAI
Track: Incident Streams
Year: 2018
Submission: 8/31/2018
Type: auto
Task: main
MD5: 7fd8a4aa8e3ac9cb7d2b7ee72f47fef6
Run description: Baseline experiment

SINAI_run2¶

Participants | Proceedings | Input | Appendix

Run ID: SINAI_run2
Participant: SINAI
Track: Incident Streams
Year: 2018
Submission: 8/31/2018
Type: auto
Task: main
MD5: fd09343a1f1459ea52aafb4310f698f6
Run description: Experiment that uses WordNet synonyms as external resource

SINAI_run3¶

Participants | Proceedings | Input | Appendix

Run ID: SINAI_run3
Participant: SINAI
Track: Incident Streams
Year: 2018
Submission: 8/31/2018
Type: auto
Task: main
MD5: 216f68e4e42584217dd980cfb73d75b6
Run description: Experiment that uses spelling correction

SINAI_run4¶

Participants | Proceedings | Input | Appendix

Run ID: SINAI_run4
Participant: SINAI
Track: Incident Streams
Year: 2018
Submission: 8/31/2018
Type: auto
Task: main
MD5: 68fd41043c41371a63def3ea6dcdc1e0
Run description: Experiment that uses word embeddings

umdhcilbaseline¶

Participants | Input | Appendix

Run ID: umdhcilbaseline
Participant: umd_hcil
Track: Incident Streams
Year: 2018
Submission: 8/29/2018
Type: auto
Task: main
MD5: ecee755bae10c23fef7a27a98b294712
Run description: This run includes a simple TF-IDF-based tokenizer that is built using a sample of Twitter data from 2013 to 2016. After vectorizing tweets, we apply a simple naive Bayes classifier to all training data across all events.

umdhcilfasttext¶

Participants | Input | Appendix

Run ID: umdhcilfasttext
Participant: umd_hcil
Track: Incident Streams
Year: 2018
Submission: 8/29/2018
Type: auto
Task: main
MD5: 8686c43c363e8e243ead97eba6464f58
Run description: This run leverages a FastText-based model trained on Twitter data from 2013-2016. After vectorizing tweets using this model, we apply a simple naive Bayes classifier to all training data across all events.

umdhcilfts¶

Participants | Input | Appendix

Run ID: umdhcilfts
Participant: umd_hcil
Track: Incident Streams
Year: 2018
Submission: 8/29/2018
Type: auto
Task: main
MD5: 73fbd45b991e35fe5cddaa794528ebac
Run description: This run leverages a the FastText model trained on Twitter data from 2013-2016 along with a subset of Twitter data from that same period that contains disaster related tokens, which we use for label propagation. After vectorizing tweets using this model, we apply a simple naive Bayes classifier to all training data across all events.

umdhcilspread¶

Participants | Input | Appendix

Run ID: umdhcilspread
Participant: umd_hcil
Track: Incident Streams
Year: 2018
Submission: 8/29/2018
Type: auto
Task: main
MD5: 0e48463600ec9d9405adbd8e4fb32c27
Run description: This run leverages a the tfidf tokenizer trained on Twitter data from 2013-2016 along with a subset of Twitter data from that same period that contains disaster related tokens, which we use for label propagation. After vectorizing tweets using this model, we apply a simple naive Bayes classifier to all training data across all events.

uogTr.R1.asp¶

Participants | Input | Appendix

Run ID: uogTr.R1.asp
Participant: uogTr
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: 2c5683eb8f8628779498d60bc5b6e42d
Run description: ASP Run 1 This run is a combined dictionary and machine learned classifier run. It uses three forms of evidence: indicator terms from the training dataset; manually identified indicator terms selected from a word embedding space; and a machine learned classifier trained on sentences from TRECTS 2013-2015 (nuggets were mapped into the information categories). Category selection is performed by majority vote across the different forms of evidence. Contents of ASP Program File: uogTr.trecis.2018.R1.asp.program { "workingDIR" : "/mnt/c/Work/Data", "sparkMode" : "local[4]", "loggingLevel" : "INFO", "modulePipeline" : [ { "unitName" : "TREC-IS 2018 Test Run 1", "unitExpanation" : "This generates a run file on the test data for the TRECIS 2018 Test dataset comprised of around 22k tweets. ", "unitType" : "batch", "unitInputReaders": [ "configurationTemplates/readers/datasets/TRECIS2018.dataset.conf" ], "unitModulePipeline" : [ "configurationTemplates/modules/batch/AttachTRECISLabels.module.conf", "configurationTemplates/modules/batch/AttachTRECISTopicInfo.module.conf", "configurationTemplates/modules/batch/PredictCategoryFromDictionary.module.conf", "configurationTemplates/modules/batch/GATEPOSEntityTagger.module.conf", "configurationTemplates/modules/batch/FactFinder.module.conf", "configurationTemplates/modules/batch/TRECISCategorySelectionLogic.module.conf", "configurationTemplates/modules/batch/ConvertToTRECISFormat.module.conf" ] } ] }

uogTr.R2.asp¶

Participants | Input | Appendix

Run ID: uogTr.R2.asp
Participant: uogTr
Track: Incident Streams
Year: 2018
Submission: 9/4/2018
Type: auto
Task: main
MD5: f25282d574fed9d7a35217899aa6b90c
Run description: ASP Run 2 Same framework as run 1, but lower the emphasis on the key terms and focus instead on the dictionary and classifier. Contents of ASP Program File: uogTr.trecis.2018.R1.asp.program { "workingDIR" : "/mnt/c/Work/Data", "sparkMode" : "local[4]", "loggingLevel" : "INFO", "modulePipeline" : [ { "unitName" : "TREC-IS 2018 Test Run 1", "unitExpanation" : "This generates a run file on the test data for the TRECIS 2018 Test dataset comprised of around 22k tweets. ", "unitType" : "batch", "unitInputReaders": [ "configurationTemplates/readers/datasets/TRECIS2018.dataset.conf" ], "unitModulePipeline" : [ "configurationTemplates/modules/batch/AttachTRECISLabels.module.conf", "configurationTemplates/modules/batch/AttachTRECISTopicInfo.module.conf", "configurationTemplates/modules/batch/PredictCategoryFromDictionary.module.conf", "configurationTemplates/modules/batch/GATEPOSEntityTagger.module.conf", "configurationTemplates/modules/batch/FactFinder.module.conf", "configurationTemplates/modules/batch/TRECISCategorySelectionLogic.module.conf", "configurationTemplates/modules/batch/ConvertToTRECISFormat.module.conf" ] } ] }

uogTr.R3.asp¶

Participants | Input | Appendix

Run ID: uogTr.R3.asp
Participant: uogTr
Track: Incident Streams
Year: 2018
Submission: 9/4/2018
Type: auto
Task: main
MD5: c05c5225f1314bd72d99cc29bcbb8d2b
Run description: ASP Run 3 Same framework as run 1, but lower ignore the classifier and indicator terms. Only use the dictionary for classification. Probably will have good precision, but will default to ContinuingNews a lot. Contents of ASP Program File: uogTr.trecis.2018.R1.asp.program { "workingDIR" : "/mnt/c/Work/Data", "sparkMode" : "local[4]", "loggingLevel" : "INFO", "modulePipeline" : [ { "unitName" : "TREC-IS 2018 Test Run 1", "unitExpanation" : "This generates a run file on the test data for the TRECIS 2018 Test dataset comprised of around 22k tweets. ", "unitType" : "batch", "unitInputReaders": [ "configurationTemplates/readers/datasets/TRECIS2018.dataset.conf" ], "unitModulePipeline" : [ "configurationTemplates/modules/batch/AttachTRECISLabels.module.conf", "configurationTemplates/modules/batch/AttachTRECISTopicInfo.module.conf", "configurationTemplates/modules/batch/PredictCategoryFromDictionary.module.conf", "configurationTemplates/modules/batch/GATEPOSEntityTagger.module.conf", "configurationTemplates/modules/batch/FactFinder.module.conf", "configurationTemplates/modules/batch/TRECISCategorySelectionLogic.module.conf", "configurationTemplates/modules/batch/ConvertToTRECISFormat.module.conf" ] } ] }

UPB_DICE1¶

Participants | Proceedings | Input | Appendix

Run ID: UPB_DICE1
Participant: DICE-UPB
Track: Incident Streams
Year: 2018
Submission: 9/3/2018
Type: auto
Task: main
MD5: 385b098239a2705b7d1fd2bc2e04abbd
Run description: We used a glove pre-trained model to represent tweets as word2vec, then we trained a deep model (two dense layers and softmax activation output) to predict information types of tweets.

UPB_DICE2¶

Participants | Proceedings | Input | Appendix

Run ID: UPB_DICE2
Participant: DICE-UPB
Track: Incident Streams
Year: 2018
Submission: 9/4/2018
Type: auto
Task: main
MD5: b8a3bb3371a1636a85bc2f266a7eb6f4
Run description: We have extracted a combined set of features from tweets(bag of words, bag of concepts and word embedding) and trained a deep model to classify the information types. The bag of concepts are provided by an external library 'Babelfly' and word embedding generated by a glove pre-trained embedding model

UPB_DICE3¶

Participants | Proceedings | Input | Appendix

Run ID: UPB_DICE3
Participant: DICE-UPB
Track: Incident Streams
Year: 2018
Submission: 9/4/2018
Type: auto
Task: main
MD5: eb8e55f3521a6ccd58d3c0796c6bca4d
Run description: For this run, we trained a support vector classifier with a linear kernel (with the penalty and tolerance parameters set to 0.1 and 0.0001, respectively) on a feature combination of 'bag of concepts' extracted from Babelfy for each tweet and 'bag of words'.

UPB_DICE4¶

Participants | Proceedings | Input | Appendix

Run ID: UPB_DICE4
Participant: DICE-UPB
Track: Incident Streams
Year: 2018
Submission: 9/4/2018
Type: auto
Task: main
MD5: d51ba2a906467070a246ec619ddd1b2d
Run description: For this run, we extracted 'bag of words' from tweets and trained it using a support vector classifier with a linear kernel (with he penalty and tolerance parameters set to 0.1 and 0.0001, respectively).