Skip to content

Proceedings - Incident Streams 2019

CBNU at TREC 2019 Incident Streams Track

Won-Gyu Choi, Kyung-Soon Lee

Abstract

This paper describes the participation of the CBNU team at the TREC Incident Streams Track 2019 [1]. Our approach is the same with CBNU at TREC-IS 2018 [2]. In our participation to TREC-IS Track 2018 and 2019, we focus on the conceptual representation for crisis-related terms. In order to classify a stream of tweets related to the incident, the terms in each tweet are represented as conceptual entities such as event entities, category indicator entities, information type entities, URL entities, and user entities. For tweet classification, we have compared support vector machines (SVM) and convolutional neural networks (CNNs).

Bibtex 
@inproceedings{DBLP:conf/trec/ChoiL19,
    author = {Won{-}Gyu Choi and Kyung{-}Soon Lee},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {{CBNU} at {TREC} 2019 Incident Streams Track},
    booktitle = {Proceedings of the Twenty-Eighth Text REtrieval Conference, {TREC} 2019, Gaithersburg, Maryland, USA, November 13-15, 2019},
    series = {{NIST} Special Publication},
    volume = {1250},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2019},
    url = {https://trec.nist.gov/pubs/trec28/papers/cbnu.IS.pdf},
    timestamp = {Wed, 03 Feb 2021 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/ChoiL19.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

ICTNET at Trec 2019 Incident Streams Track

Guangsheng Kuang, Kun Zhang, Jiabao Zhang, Xin Zheng

Abstract

Social medial become our public ways to share our information in our lives. Crisis management via social medial is becoming indispensable for its tremendous information. While deep learning shows surprising outcome in many tasks. So in this paper, we cope this learning task with neural network in the view of classification problem.

Bibtex 
@inproceedings{DBLP:conf/trec/GuangshengKJX19,
    author = {Guangsheng Kuang and Kun Zhang and Jiabao Zhang and Xin Zheng},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {{ICTNET} at Trec 2019 Incident Streams Track},
    booktitle = {Proceedings of the Twenty-Eighth Text REtrieval Conference, {TREC} 2019, Gaithersburg, Maryland, USA, November 13-15, 2019},
    series = {{NIST} Special Publication},
    volume = {1250},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2019},
    url = {https://trec.nist.gov/pubs/trec28/papers/ICTNET.IS.pdf},
    timestamp = {Wed, 03 Feb 2021 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/GuangshengKJX19.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

Anna M. Kruspe, Jens Kersten, Friederike Klan

Abstract

In this paper, we present our five approaches submitted to the Text REtrieval Conference (TREC) Incident Streams (IS) 2019B edition. The goal is to classify crisis-related tweets into a variable set of information classes and to provide an importance score. This multi-class, multi-label and multi-task problem turns out to be even more challenging because of extremely unbalanced training data available. We use recently proposed, publicy available word and sentence embeddings and deep neural network models for this task.

Bibtex 
@inproceedings{DBLP:conf/trec/KruspeKK19,
    author = {Anna M. Kruspe and Jens Kersten and Friederike Klan},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {Classification of Incident-related Tweets: Exploiting Word and Sentence Embeddings},
    booktitle = {Proceedings of the Twenty-Eighth Text REtrieval Conference, {TREC} 2019, Gaithersburg, Maryland, USA, November 13-15, 2019},
    series = {{NIST} Special Publication},
    volume = {1250},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2019},
    url = {https://trec.nist.gov/pubs/trec28/papers/DLR\_DW.IS.pdf},
    timestamp = {Wed, 03 Feb 2021 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/KruspeKK19.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

IIT BHU at TREC 2019 Incident Streams Track

Akanksha Mishra, Sukomal Pal

Abstract

The paper describes the participation of the IIT BHU at the TREC 2019B Incident Streams track. We submitted two fully automatic runs for categorizing information within tweet into multiple high-level information types and determining the criticality score for each tweet given in the test set.

Bibtex 
@inproceedings{DBLP:conf/trec/MishraP19,
    author = {Akanksha Mishra and Sukomal Pal},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {{IIT} {BHU} at {TREC} 2019 Incident Streams Track},
    booktitle = {Proceedings of the Twenty-Eighth Text REtrieval Conference, {TREC} 2019, Gaithersburg, Maryland, USA, November 13-15, 2019},
    series = {{NIST} Special Publication},
    volume = {1250},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2019},
    url = {https://trec.nist.gov/pubs/trec28/papers/IIT\_BHU.IS.pdf},
    timestamp = {Wed, 03 Feb 2021 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/MishraP19.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

Congcong Wang, David Lillis

Abstract

This paper presents University College Dublin's (UCD) work at TREC 2019-B Incident Streams (IS) track. The purpose of the IS track is to find actionable messages and estimate their priority among a stream of crisis-related tweets. Based on the track's requirements, we break down the task into two sub-tasks. One is defined as a multi-label classification task that categorises upcoming tweets into different aid requests. The other is defined as a single-label classification task that estimates these tweets with four different levels of priority. For the track, we submitted four runs, each of which uses a different model for the tasks. Our baseline run trains classification models with hand-crafted features through machine learning methods, namely Logistic Regression and Naïve Bayes. Our other three runs train classification models with different deep learning methods. The deep methods include a vanilla bidirectional long short-term memory recurrent neural network (biLSTM), an adapted biLSTM, and a bi-attentive classification network (BCN) with pre-trained contextualised ELMo embedding. For all the runs, we apply different word embeddings (in-domain pre-trained, word-level pre-trained GloVe, character-level, or ELMo embeddings) and data augmentation strategies (SMOTE, loss weights, or GPT-2) to explore the influence they have on performance. Evaluation results show that our models perform better than the median for most situations.

Bibtex 
@inproceedings{DBLP:conf/trec/WangL19,
    author = {Congcong Wang and David Lillis},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {Classification for Crisis-Related Tweets Leveraging Word Embeddings and Data Augmentation},
    booktitle = {Proceedings of the Twenty-Eighth Text REtrieval Conference, {TREC} 2019, Gaithersburg, Maryland, USA, November 13-15, 2019},
    series = {{NIST} Special Publication},
    volume = {1250},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2019},
    url = {https://trec.nist.gov/pubs/trec28/papers/CS-UCD.IS.pdf},
    timestamp = {Wed, 03 Feb 2021 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/WangL19.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

Fine-tuned BERT Model for Multi-Label Tweets Classification

Hamada M. Zahera, Ibrahim A. Elgendy, Rricha Jalota, Mohamed Ahmed Sherif

Abstract

In this paper, we describe our approach to classify disaster-related tweets into multi-label information types (i.e, labels). We aim to filter first relevant tweets during disasters. Then, we assign tweets relevant information types. Information types can be SearchAndRescue, MovePeople and Volunteer. We employ a fine-tuned BERT model with 10 BERT layers. Further, we submitted our approach to the TREC-IS 2019 challenge, the evaluation results showed that our approach outperforms the F1-score of median score in identifying actionable information.

Bibtex 
@inproceedings{DBLP:conf/trec/ZaheraEJS19,
    author = {Hamada M. Zahera and Ibrahim A. Elgendy and Rricha Jalota and Mohamed Ahmed Sherif},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {Fine-tuned {BERT} Model for Multi-Label Tweets Classification},
    booktitle = {Proceedings of the Twenty-Eighth Text REtrieval Conference, {TREC} 2019, Gaithersburg, Maryland, USA, November 13-15, 2019},
    series = {{NIST} Special Publication},
    volume = {1250},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2019},
    url = {https://trec.nist.gov/pubs/trec28/papers/DICE\_UPB.IS.pdf},
    timestamp = {Wed, 03 Feb 2021 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/ZaheraEJS19.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}

CMU-Informedia at TREC 2019 Incident Streams Track

Junpei Zhou, Xinyu Wang, Po-Yao Huang, Alexander G. Hauptmann

Abstract

We describe CMU-Informedia's models for the TREC 2019 Incident Streams track. The goal of this track is classifying event/incident related tweets by High-level Information Types such as 'SearchAndRescue', 'InformationWanted' and so on. Each tweet should be assigned as many categories as are appropriate. What's more, this track requires predicting the Importance Scores, which is converted from the Importance Labels including 'Critical', 'High', 'Medium', 'Low' and 'Irrelevant'. For predicting the information types, we use feature extractors to extract features including meta-information, user entity, and textual embeddings, and then we build an information type predictor on those features. For predicting the importance scores, we build an importance score predictor which combines the scores derived from the predicted information types and the scores produced by a regression model. Evaluation results show that our models perform well on all metrics, and different models perform particularly well on different aspects.

Bibtex 
@inproceedings{DBLP:conf/trec/ZhouWHH19,
    author = {Junpei Zhou and Xinyu Wang and Po{-}Yao Huang and Alexander G. Hauptmann},
    editor = {Ellen M. Voorhees and Angela Ellis},
    title = {CMU-Informedia at {TREC} 2019 Incident Streams Track},
    booktitle = {Proceedings of the Twenty-Eighth Text REtrieval Conference, {TREC} 2019, Gaithersburg, Maryland, USA, November 13-15, 2019},
    series = {{NIST} Special Publication},
    volume = {1250},
    publisher = {National Institute of Standards and Technology {(NIST)}},
    year = {2019},
    url = {https://trec.nist.gov/pubs/trec28/papers/CMUInformedia.IS.pdf},
    timestamp = {Wed, 03 Feb 2021 00:00:00 +0100},
    biburl = {https://dblp.org/rec/conf/trec/ZhouWHH19.bib},
    bibsource = {dblp computer science bibliography, https://dblp.org}
}