Runs - Legal 2008¶

adhocpool08¶

Results | Participants | Proceedings | Appendix

Run ID: adhocpool08
Participant: ot
Track: Legal
Year: 2008
Submission: 9/10/2008
Task: interactive
MD5: 5ad7473cf5f101923ab38a46b6c525c9
Run description: This run is the full depth-100000 pool of all submitted Ad Hoc runs for topics 102, 103 and 104, including the reference Boolean runs (but not including the randomL08 run).

Clearwell08¶

Results | Participants | Appendix

Run ID: Clearwell08
Participant: Clearwell
Track: Legal
Year: 2008
Submission: 9/11/2008
Task: interactive
MD5: 6b6dcf739d4d1481f5652beac8c3b423
Run description: We used several techniques for mining the data. The CDIP collection was processed using Perl and other Unix scripts to cleanse the content. We then used Clearwell E-Discovery Platform and other third-party eDiscovery packages to extract documents based on keywords, boolean search, fuzzy search and concept search. We then used iterative searching techniques driven by human review. Quality control was based on a sample (95% confidence level) with review of both the non-responsive collection and responsive collection, followed by native export. The final stage was to consolidate and clean the submission using another set of Unix scripts and Perl scripts.

CTFgge10kBr0¶

Run ID: CTFgge10kBr0
Participant: WIR
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: manual
Task: adhoc
MD5: 16b1ee4db78f30ec1fb70dc128da93a2
Run description: We submit automatically constructed weighted queries obtained indexing the following topic fields - request text - initial proposal by defendant - rejoinder by plaintiff - final negotiated Boolean query ... and appling a semi-automatically built stop word list. K constant (k=100.000).

CTFgge4kBr0¶

Run ID: CTFgge4kBr0
Participant: WIR
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: manual
Task: adhoc
MD5: 8180c03927feb11c68b4339b707241b0
Run description: We submit automatically constructed weighted queries obtained indexing the following topic fields - request text - initial proposal by defendant - rejoinder by plaintiff - final negotiated Boolean query ... and appling a semi-automatically built stop word list. K constant (k=40.000) with boolean re-rank.

CTFggeBkBr0¶

Run ID: CTFggeBkBr0
Participant: WIR
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: manual
Task: adhoc
MD5: 4854936a6becba1daf089a7de7ae173b
Run description: We submit automatically constructed weighted queries obtained indexing the following topic fields - request text - initial proposal by defendant - rejoinder by plaintiff - final negotiated Boolean query ... and appling a semi-automatically built stop word list. K based on B values, with boolean re-rank.

CTFggeBkBr1¶

Run ID: CTFggeBkBr1
Participant: WIR
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: manual
Task: adhoc
MD5: 135f1ade73826b73a2f00baa61186f1f
Run description: We submit automatically constructed weighted queries obtained indexing the following topic fields - request text - initial proposal by defendant - rejoinder by plaintiff - final negotiated Boolean query ... and appling a semi-automatically built stop word list. K based on B values, with a different factor for boolean re-rank.

CTFggeRkBr0¶

Run ID: CTFggeRkBr0
Participant: WIR
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: manual
Task: adhoc
MD5: e784cc7478ecad9fc12bc5a8d0d82ce9
Run description: We submit automatically constructed weighted queries obtained indexing the following topic fields - request text - initial proposal by defendant - rejoinder by plaintiff - final negotiated Boolean query ... and appling a semi-automatically built stop word list. K=B with boolean re-rank.

CTFggeSkBr0¶

Run ID: CTFggeSkBr0
Participant: WIR
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: manual
Task: adhoc
MD5: 2982dd22a6c364f1abd83c6c8a3de22c
Run description: We submit automatically constructed weighted queries obtained indexing the following topic fields - request text - initial proposal by defendant - rejoinder by plaintiff - final negotiated Boolean query ... and appling a semi-automatically built stop word list. K computed also considering best values of K at Legal track 2007, with boolean re-rank.

CTFrtSk¶

Run ID: CTFrtSk
Participant: WIR
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 1693884962779f6b4878b6180483ce25
Run description: Baseline using the Terrier Framework with a parameter free IR model.

CTFrtSkBr0¶

Run ID: CTFrtSkBr0
Participant: WIR
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 542bcf729352839f320a83c47af49834
Run description: We re-rank the baseline using the boolean reference run.

H52008¶

Results | Participants | Proceedings | Appendix

Run ID: H52008
Participant: H5_2008
Track: Legal
Year: 2008
Submission: 9/12/2008
Task: interactive
MD5: 39f00907cc9c65eb283872d27fe5d8bc
Run description: H5 has submitted an assessment of documents our system has identified as responsive to topic 103 of TREC's Legal Track Interactive Task a total of 608807 documents. The H5 system combined human expertise and advanced technologies to assess the totality of the corpus under investigation.

IowaSL0804¶

Run ID: IowaSL0804
Participant: UIowa-Srinivasan
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: fda369ec9dd1917f3bf6efd91d71701f
Run description: The source fields are converted to a term vector and submitted as a query. Five terms are extracted from the top three documents returned from an initial query. These terms are then added to the term vector for the final query. Documents are given a score and ranked as per IowaSL08Ref.

IowaSL0804b¶

Run ID: IowaSL0804b
Participant: UIowa-Srinivasan
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 97d33ba8e222dfe684c4791284e34dfd
Run description: Documents are retrieved and ranked in the same manner as IowaSL0804. Additionally, documents that were also found by the Boolean Reference run provided by the track organizers are given a boost to their score, and the set is then reranked. The boost, a multiplier of 1.8, was found through training on 2007 results. K-values are taken from B, and Kh is B/2.

IowaSL0805¶

Run ID: IowaSL0805
Participant: UIowa-Srinivasan
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 79f97ce510a5aadb16b54d47314622cb
Run description: Documents are retrieved and ranked as in IowaSL0804 except that, prior to being used in initial or final queries, the Request Text is automatically modified to remove terms that appear in too many instances of this field across all the topics.

IowaSL0805b¶

Run ID: IowaSL0805b
Participant: UIowa-Srinivasan
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 448e2f145c3fcbbdd69f8462757c662a
Run description: Documents are retrieved and ranked in the same manner as IowaSL0804. Additionally, documents that were also found by the Boolean Reference run provided by the track organizers are given a boost to their score, and the set is then reranked. The boost, a multiplier of 1.5, was found through training on 2007 results. K-values are taken from B, and Kh is B/2.

IowaSL0808b¶

Run ID: IowaSL0808b
Participant: UIowa-Srinivasan
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: b7185230ba699b7065fbfb2b69a1ab16
Run description: Documents are retrieved and ranked in the same manner as IowaSL0804b, except that the entirety of the negotiation history is also included as a source of terms for the query term vectors.

IowaSL0808m2¶

Run ID: IowaSL0808m2
Participant: UIowa-Srinivasan
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 0ed1eaa5904016a3d687f58d1c1a585d
Run description: The documents submitted for this run were found by fusing the results from IowaSL0804b and IowaSL0805b. The runs were weighted 1.0 and 0.755 respectively, in accordance to the relative performance of similar runs submitted to the TREC 2007 Legal Track ad hoc task, using the F1@R measure.

IowaSL0808m3¶

Run ID: IowaSL0808m3
Participant: UIowa-Srinivasan
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 032c60e02a592349e5962929d2d09480
Run description: The documents submitted for this run were found by fusing the results from IowaSL0804b, IowaSL0805b, and IowaSL0808b. The runs were weighted 2.0, 1.0, and 3.0 respectively. The rationale for the weighting is to grant more weight to runs with more information from the topic source fields contributing to the run queries.

IowaSL08Ref¶

Run ID: IowaSL08Ref
Participant: UIowa-Srinivasan
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 59f71590f2b685ed6abad7a330cc4ae5
Run description: This is the requested standard run. The source field is converted to a term vector and submitted as a query. Documents are given a score according to the Okapi-BM25 function with parameters as described in our paper for the TREC Legal Track 2007; documents are ranked based on this score. K-values are taken from B, and Kh is B/2.

IowaSL08RF1A¶

Run ID: IowaSL08RF1A
Participant: UIowa-Srinivasan
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: feedback
MD5: 132813034fe23d7e70cc41129da59b56
Run description: A collection of features is calculated for all documents submitted in the source runs for IowaSL08RF1B, judged and unjudged. The features describe the position of the document in the contributing runs, as well as the relevance characteristics of its "neighborhood". We train a classifier using the calculated features of the judged documents. The classifier, implemented using Wekas SMO package, fits a logit regression to generate probabilities for relevance and nonrelevance for each of the test documents. It is then applied to the unjudged, which are then ranked by estimated probability of relevance.

IowaSL08RF1B¶

Run ID: IowaSL08RF1B
Participant: UIowa-Srinivasan
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: feedback
MD5: 17bac6e99ccf633548ddec35d724ca84
Run description: Each submitted run for the previous ad hoc tasks were pooled to build a collection of judged and unjudged documents. The unjudged documents are merged, ranked using a score reflecting a summation of the ranks across the runs and a multiplier reflecting the number of runs to select the document (CombMNZ). For the 2007 topics, the 101000 documents with the best scores are selected.

IowaSL08RF2A¶

Run ID: IowaSL08RF2A
Participant: UIowa-Srinivasan
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: feedback
MD5: affe1e85ef4bf2c04b1ad257fb857d45
Run description: This run is processed in the same manner IowaSL08RF1A. The documents pooled for IowaSL08RF2B are split into training and evaluation pools in the same manner as described for IowaSL08RF1A.

IowaSL08RF2B¶

Run ID: IowaSL08RF2B
Participant: UIowa-Srinivasan
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: feedback
MD5: 3c8419f735a35bd3fb48e4be64dc4a9a
Run description: This run is processed in the same manner IowaSL08RF1B, the difference laying in the runs contributing to the pool of judged and unjudged documents. The seven runs submitted by the IowaS group for the TREC Legal 2007 Ad Hoc task are used for the subset of 2007 topics. Results generated Post TREC 2007 by applying 2007 strategies against the 2006 topics are used as sources for the 2006 topics.

IowaSL08RF2C¶

Run ID: IowaSL08RF2C
Participant: UIowa-Srinivasan
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: feedback
MD5: c97ebf13229b91be3f25f5d0dc184c39
Run description: This run is also processed in the same manner IowaSL08RF2A. The documents pooled for IowaSL08RF2B are split into training and evaluation pools in the same manner as described for IowaSL08RF1A. For this run, the training pool is discarded, and the classifiers built using all submitted runs, for run IowaSL08RF1A, are used for classifying and ranking the evaluation pool.

IowaSL08RF3A¶

Run ID: IowaSL08RF3A
Participant: UIowa-Srinivasan
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: feedback
MD5: 785155108a9e6ace99b0361f5890d7bf
Run description: This run is also processed in the same manner IowaSL08RF1A. The documents pooled for IowaSL08RF3B are split into training and evaluation pools in the same manner as described for IowaSL08RF1A.

IowaSL08RF3B¶

Run ID: IowaSL08RF3B
Participant: UIowa-Srinivasan
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: feedback
MD5: d20054ef308dbabb2a6d549e014b1b32
Run description: This run is processed in the same manner IowaSL08RF1B, the difference laying in the runs contributing to the pool of judged and unjudged documents. For 2006 topics, four runs with leading performance on the R-Prec measure across all topics were selected, and four runs with poorer performance were selected. A similar technique was used to identify contributing runs for the 2007 topics, using the Est_Recall@B measure.

IowaSL08RFTr¶

Run ID: IowaSL08RFTr
Participant: UIowa-Srinivasan
Track: Legal
Year: 2008
Submission: 8/7/2008
Type: automatic
Task: feedback
MD5: 386d6789b26d3a85addf0d2637229e63
Run description: This run is run in the same manner as our ad hoc run IowaSL0804 with the following modifications to take advantage of relevance feedback. Documents that have not been judged to be relevant are deemed ineligible for providing terms for query expansion. Training has allowed us to expand the number of query expansion terms to 12 from up to 10 relevant documents. The boolean reference run boost has not been applied. K and Kh values are from R and R/3 respectively.

otL08db¶

Run ID: otL08db
Participant: ot
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: manual
Task: adhoc
MD5: c9b60e4a20bf8a7c5998a4858fc2c6dd
Run description: a ranked Boolean run based on the initial proposal by defendant (same as otL08fb except that the defendant Boolean query was used instead of the final query)

otL08fb¶

Run ID: otL08fb
Participant: ot
Track: Legal
Year: 2008
Submission: 7/31/2008
Type: manual
Task: adhoc
MD5: 6ef71be6eacef02259aa4b77b8afc5e4
Run description: Boolean run based on final negotiated query (same as reference Boolean run refL08B except re-ordered by relevance)

otL08fbe¶

Run ID: otL08fbe
Participant: ot
Track: Legal
Year: 2008
Submission: 8/2/2008
Type: manual
Task: adhoc
MD5: a21ab560ea0652a647bdedaf3e577228
Run description: blind feedback expansion of the final Boolean query, ranked-based fusion, weight 2 on original otL08fb query and weight 1 on each of first 2 retrieved rows of otL08fb as query; K includes weight sum of 2 or more, Kh includes weight sum of 3 or more

otL08frw¶

Run ID: otL08frw
Participant: ot
Track: Legal
Year: 2008
Submission: 8/2/2008
Type: manual
Task: adhoc
MD5: d98d678cd56d844d0605589510f1df1e
Run description: weighted rank-based fusion of Boolean and vector runs, weight 3 on otL08fb, weight 2 on otL08rvl, weight 1 on otL08fv; K includes weight sum of 3 or more, Kh includes weight sum of 4 or more

otL08fv¶

Run ID: otL08fv
Participant: ot
Track: Legal
Year: 2008
Submission: 8/2/2008
Type: manual
Task: adhoc
MD5: 74d4406d7ea8ea506851e244955be185
Run description: vector run based on final negotiated query (same as otL08fb except that all Boolean, phrase and proximity operators were changed to the OR operator); K includes rsv of 200 or more, Kh includes rsv of 225 or more

otL08rv¶

Run ID: otL08rv
Participant: ot
Track: Legal
Year: 2008
Submission: 8/3/2008
Type: manual
Task: adhoc
MD5: fee085b46516f66c234b53b1d600b7a9
Run description: vector run based on request text terms (same as otL08rvl except no stemming, i.e. English inflections were not matched)

otL08rvl¶

Run ID: otL08rvl
Participant: ot
Track: Legal
Year: 2008
Submission: 8/2/2008
Type: manual
Task: adhoc
MD5: 68d7173eaff1232ac3a5b3c333a64ac3
Run description: vector run based on request text terms (same as otL08fv except (1) the terms were taken from the request text field instead of the final Boolean field, (2) English inflections were matched, (3) common instruction words (e.g. "please", "produce", "documents") were manually removed); K includes rsv of 200 or more, Kh includes rsv of 225 or more

otL08rvlq¶

Run ID: otL08rvlq
Participant: ot
Track: Legal
Year: 2008
Submission: 8/3/2008
Type: manual
Task: adhoc
MD5: ebf92d2a38be57347de074bdd10feb2a
Run description: vector run based on request text terms (same as otL08rvl except that common instruction words (e.g. "please", "produce", "documents") were not removed)

otRF08F¶

Run ID: otRF08F
Participant: ot
Track: Legal
Year: 2008
Submission: 8/5/2008
Type: manual
Task: feedback
MD5: 65a95358d3f60713b7e1909da0d67b0b
Run description: pure relevance feedback run based on forming a query from the known relevant documents (excluding those larger than 1 million bytes); no topic fields were used; K includes rsv of 200 or more, Kh includes rsv of 225 or more

otRF08fb¶

Run ID: otRF08fb
Participant: ot
Track: Legal
Year: 2008
Submission: 8/4/2008
Type: manual
Task: feedback
MD5: 8eb6ecc299ff67700754dcda997bd176
Run description: Boolean run based on final negotiated query (same as reference Boolean run refRF08B except re-ordered by relevance)

otRF08fbF¶

Run ID: otRF08fbF
Participant: ot
Track: Legal
Year: 2008
Submission: 8/5/2008
Type: manual
Task: feedback
MD5: 3f1f5049db57bf467db3259551e635a5
Run description: ranked-based fusion of Boolean (otRF08fb) and feedback (otRF08F) runs, equal weights; K and Kh are the same as for otRF08F

otRF08fbFR¶

Run ID: otRF08fbFR
Participant: ot
Track: Legal
Year: 2008
Submission: 8/7/2008
Type: manual
Task: feedback
MD5: d52e41bbede5c3b6442b99cd72e75297
Run description: same documents as otRF08fbF and same K and Kh as otRF08FR

otRF08FR¶

Run ID: otRF08FR
Participant: ot
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: manual
Task: feedback
MD5: f8dbe07323d710a6c62b5b9e5c77e1b5
Run description: same as otRF08F except that K is based on estimated R from humL06tvz sample for first 15 topics and L07 est_rel for latter 25 topics; Kh same as K

otRF08frw¶

Run ID: otRF08frw
Participant: ot
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: manual
Task: feedback
MD5: 192ea4ffe69b61778acf3497228de674
Run description: weighted rank-based fusion of Boolean and vector runs (same approach as otL08frw, i.e. weight 3 on otRF08fb, weight 2 on otRF08rvl, weight 1 on otRF08fv; K includes weight sum of 3 or more, Kh includes weight sum of 4 or more)

otRF08fv¶

Run ID: otRF08fv
Participant: ot
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: manual
Task: feedback
MD5: 475d6eb7fc07bd1039f11b29d06e1943
Run description: vector run based on final negotiated query terms (same approach as otL08fv)

otRF08rvl¶

Run ID: otRF08rvl
Participant: ot
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: manual
Task: feedback
MD5: 812cf06ecac5a30b79e4067030817e0b
Run description: vector run based on request text terms (same approach as otL08rvl)

PittSIST1¶

Results | Participants | Proceedings | Appendix

Run ID: PittSIST1
Participant: Pitt_IR_Team
Track: Legal
Year: 2008
Submission: 9/11/2008
Task: interactive
MD5: 4433a59396c1580b202aa5ede7f3d3be
Run description: We have around 5,000 documents for T102 and around 32,000 documents for T103.

RMITbp1¶

Run ID: RMITbp1
Participant: rmit
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: c5ba3f029132b3f3d72716fb7d18d13b
Run description: Boolean ranked search on OCR fields.

RMITbp2¶

Run ID: RMITbp2
Participant: rmit
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: a82f2e85e6eaf5a36ac813f34c9ada57
Run description: Boolean ranked search on the processed OCR field (removed HTML entities and soft hyphens).

RMITbp3¶

Run ID: RMITbp3
Participant: rmit
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: f3cc77c26d8305f9000a30677e0f753b
Run description: Boolean ranked search on processed OCR fields (removed HTML entities, soft hyphens, and low frequency terms).

RMITrp1¶

Run ID: RMITrp1
Participant: rmit
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: cb115377c82cc319d67fb966649e2310
Run description: Request text search on OCR field.

RMITrp2¶

Run ID: RMITrp2
Participant: rmit
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 1bcf6cd0544a01a539e763d23c4e5824
Run description: Request text search on the processed OCR field.(removed HTML entities and soft hyphens).

RMITrp3¶

Run ID: RMITrp3
Participant: rmit
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: c41b18dace5e713e0b984417da1f61d3
Run description: Request text search on the processed OCR field (removed HTML entities, soft hyphens, and low frequency terms).

SabL08ab1¶

Results | Participants | Summary (adhoc) | Summary (adhocH) | Appendix

Run ID: SabL08ab1
Participant: sabir.buckley
Track: Legal
Year: 2008
Submission: 8/7/2008
Type: automatic
Task: adhoc
MD5: f906359b43f3777a7de1d70b57e1c28b
Run description: Basic blind feedback, top 30 docs, add 20terms. SMART ltu_Lnu vector run. default constant K. Used all boolean query negotiation terms

SabL08ar2¶

Results | Participants | Summary (adhoc) | Summary (adhocH) | Appendix

Run ID: SabL08ar2
Participant: sabir.buckley
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: e8df102b8938e5bc5d67ff383714c282
Run description: Very basic SMART ltu_Lnu vector run. default constant K.

SabL08arbn¶

Results | Participants | Summary (adhoc) | Summary (adhocH) | Appendix

Run ID: SabL08arbn
Participant: sabir.buckley
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 624bd96419d0fc229ec42ce9fc969b30
Run description: Very basic SMART ltu_Lnu vector run. default constant K. Used all boolean query negotiation terms

SabL08rf1¶

Results | Participants | Summary (resid) | Summary (residH) | Appendix

Run ID: SabL08rf1
Participant: sabir.buckley
Track: Legal
Year: 2008
Submission: 8/7/2008
Type: automatic
Task: feedback
MD5: d1151356dfa14eaa255ad3930301584c
Run description: Basic rocchio feedback, all judged docs, add 40terms. SMART ltu_Lnu vector run. default constant K. Used all boolean query negotiation terms

SabL08rfbase¶

Results | Participants | Summary (resid) | Summary (residH) | Appendix

Run ID: SabL08rfbase
Participant: sabir.buckley
Track: Legal
Year: 2008
Submission: 8/7/2008
Type: automatic
Task: feedback
MD5: 9aa0d5dccc9ecf5920c2e81d47e790f2
Run description: Baseline for Rocchiofeedback, SMART ltu_Lnu vector run. default constant K. Used all boolean query negotiation terms

ublegal08¶

Results | Participants | Proceedings | Appendix

Run ID: ublegal08
Participant: SUNY_Buffalo
Track: Legal
Year: 2008
Submission: 9/12/2008
Task: interactive
MD5: 7b821d8b3fb3c57f534b171f9dc9570d
Run description: We used the online search system of UCSF Legacy Tobacco Documents Library. Our main effort was on query formulation. We tried many queries, read many retrieved documents, and had quite a few clarification emails with the topic authority. Our submitted results contain documents retrieved using four separate Boolean queries. Each of the four queries aims at certain groups of responsive documents, and we feel together they give better search results in terms of both recall and precision. No degree of relevance is implied in the order of the submitted documents because the system is a pure Boolean system.

UCBM25T10Th5¶

Run ID: UCBM25T10Th5
Participant: Ursinus_College
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: feedback
MD5: 622ff82f25f18fadacfcc21e19e22013
Run description: BM25 with 10 terms from judged rel docs added to each query (as long as the term weight was at least 5)

UCBM25T5Th5¶

Run ID: UCBM25T5Th5
Participant: Ursinus_College
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: feedback
MD5: a5a91244ab1114f284c5a2d7ad552fcb
Run description: BM25 with automatic rel feedback using top 5 terms from all relevant docs (with term weight > 5)

UCEDLSIa¶

Run ID: UCEDLSIa
Participant: Ursinus_College
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 3166f03fd9bd164fef86dc75c5f35ac1
Run description: Distributed EDLSI with lsi_k=41 and x=.2. no folding up.

UCEDLSIb¶

Run ID: UCEDLSIb
Participant: Ursinus_College
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 29ff5174e6bb91aaeb14f737bc948c37
Run description: EDLSI run with LSI_k =10 and x=.2

UCPwrT10Th5¶

Run ID: UCPwrT10Th5
Participant: Ursinus_College
Track: Legal
Year: 2008
Submission: 8/7/2008
Type: automatic
Task: feedback
MD5: bd5210e6a209fcfa33f4c1b8baeae751
Run description: Power Norm with auto rel feedback, the top 10 terms from any document judged relevant.

UCPwrT5Th5¶

Run ID: UCPwrT5Th5
Participant: Ursinus_College
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: feedback
MD5: 0f3531e6136b31fb24087fcb766cc37c
Run description: Power Norm with 5 terms added to each query. Top 5 weighted terms from any relevant document.

UCRFBM25BL¶

Run ID: UCRFBM25BL
Participant: Ursinus_College
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: feedback
MD5: 17937895390fbc9b268f688f28a6b2b1
Run description: BM25 baseline for comparison with BM25T5Th5 and BM25T10Th5

UCRFPwrBL¶

Run ID: UCRFPwrBL
Participant: Ursinus_College
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: feedback
MD5: 7faf4e2dbd3d8876f8e1ded0f2d0c23e
Run description: Power Norm baseline for UCPwrT5Th5 and UCPwrT10Th5

UIowa08Leg3¶

Results | Participants | Summary (adhoc) | Summary (adhocH) | Appendix

Run ID: UIowa08Leg3
Participant: UI-ICTS
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: e87a2693e999fa4f42f423914df3778d
Run description: Boolean query processed with standard query analyzer. This is then as pseudo-relevance feedback (top 10 documents) to a modified query that includes authors, mentions, recipients and prodbox.

UIowa08Lega¶

Results | Participants | Summary (adhoc) | Summary (adhocH) | Appendix

Run ID: UIowa08Lega
Participant: UI-ICTS
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: f84666013e1c37dbd15fa79f4cc6700c
Run description: boolean query processed with standard query analyzer - NOTE - this is actually a corrected version of UIowa08Leg1, which was accidently run with the UIowa08Leg2 configuraiton.

UIowa08LegE0¶

Results | Participants | Summary (adhoc) | Summary (adhocH) | Appendix

Run ID: UIowa08LegE0
Participant: UI-ICTS
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: e0526aac47789a295ebef6d5a31762f2
Run description: required request run

UIowa08LegE1¶

Results | Participants | Summary (adhoc) | Summary (adhocH) | Appendix

Run ID: UIowa08LegE1
Participant: UI-ICTS
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 865a4c0a9ddae60c65f34204cbb7d1e5
Run description: boolean query processed with standard query analyzer

UIowa08LegE2¶

Results | Participants | Summary (adhoc) | Summary (adhocH) | Appendix

Run ID: UIowa08LegE2
Participant: UI-ICTS
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: f552b7f8998b0dbbeeabdcf61911b714
Run description: boolean query processed with custom query analyzer handling complex proximity expressions

UIowa08LegE4¶

Results | Participants | Summary (adhoc) | Summary (adhocH) | Appendix

Run ID: UIowa08LegE4
Participant: UI-ICTS
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: a8554cec31dee9ad6a5d66568a31d8c8
Run description: Boolean query processed with custom query analyzer handling complex proximity expressions. This is then as pseudo-relevance feedback (top 10 documents) to a modified query that includes authors, mentions, recipients and prodbox.

UMDAURCC40¶

Run ID: UMDAURCC40
Participant: UMCP
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 70c74fc65c013df438173e7b195760f9
Run description: Query construction uses request text, initial proposal by defendent, rejoinder by plaintiff, and final negotiated Boolean query. We did one iteration of blind relevance feedback using metadata expansion. We substituted a portion of the boolean run using a portion of our run.

UMDAURCP3¶

Run ID: UMDAURCP3
Participant: UMCP
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 7e33ec12084467061d572c0c89d9a80b
Run description: Query construction uses request text, initial proposal by defendent, rejoinder by plaintiff, and final negotiated Boolean query. We did one iteration of blind relevance feedback using metadata expansion. Then combined with boolean results in a hill climbing manner.

UMDCRC40¶

Run ID: UMDCRC40
Participant: UMCP
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 9b40afcfda932132a7f3591b4dc0ec9a
Run description: Query construction uses request text, initial proposal by defendent, rejoinder by plaintiff, and final negotiated Boolean query. We substituted a portion of the boolean run using a portion of our run. No blind feedback is done.

UMDCRP3¶

Run ID: UMDCRP3
Participant: UMCP
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 639326ff1afc7e04cbf604ee2d3cff64
Run description: Query construction uses request text, initial proposal by defendent, rejoinder by plaintiff, and final negotiated Boolean query. Combined with boolean results in a hill climbing manner.

UMDSTD¶

Run ID: UMDSTD
Participant: UMCP
Track: Legal
Year: 2008
Submission: 8/4/2008
Type: automatic
Task: adhoc
MD5: 995739c28bc39e5af5984a26911606ad
Run description: This is the required standard condition run.

UMKCTL08RF1¶

Results | Participants | Summary (resid) | Summary (residH) | Appendix

Run ID: UMKCTL08RF1
Participant: UMKCSCE
Track: Legal
Year: 2008
Submission: 8/3/2008
Type: manual
Task: feedback
MD5: 2c21ed265925318161d84cd005b115a5
Run description: TL08RF VSM Expand15 Ratio1 SPDL719-719_0.8 101k

UMKCTL08RF2¶

Results | Participants | Summary (resid) | Summary (residH) | Appendix

Run ID: UMKCTL08RF2
Participant: UMKCSCE
Track: Legal
Year: 2008
Submission: 8/3/2008
Type: manual
Task: feedback
MD5: fd9d2a008b7e7f6110411b92e6fa61d2
Run description: TL08RF BM25 Expand15 Ratio1 s0.9k3 101k K1000

UMKCTL08RF3¶

Results | Participants | Summary (resid) | Summary (residH) | Appendix

Run ID: UMKCTL08RF3
Participant: UMKCSCE
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: manual
Task: feedback
MD5: a18be2eb096c7829d3d2d00655e1d9eb
Run description: UMKC TL08RF LM_0.2-1.0 Expand15 Ratio1 101k K1000

UMKCTL08RF5¶

Results | Participants | Summary (resid) | Summary (residH) | Appendix

Run ID: UMKCTL08RF5
Participant: UMKCSCE
Track: Legal
Year: 2008
Submission: 8/7/2008
Type: manual
Task: feedback
MD5: 990669981769c01993081bbb7906ac22
Run description: CombMNZ Rank Fusion Normalized Score refRF08B.append

UMKCTL08RF6¶

Results | Participants | Summary (resid) | Summary (residH) | Appendix

Run ID: UMKCTL08RF6
Participant: UMKCSCE
Track: Legal
Year: 2008
Submission: 8/7/2008
Type: manual
Task: feedback
MD5: a163ad4e565a143d3d1a0224420f765d
Run description: CombMNZ Rank Fusion Normalized Score with refRF08B refRF08B.append

UrsinusBM25a¶

Run ID: UrsinusBM25a
Participant: Ursinus_College
Track: Legal
Year: 2008
Submission: 8/2/2008
Type: automatic
Task: adhoc
MD5: 7d7385a29594e795af72b0bc23f79afd
Run description: BM25 baseline optimized using TREC 06 and 07 queries for training

UrsinusBM25b¶

Run ID: UrsinusBM25b
Participant: Ursinus_College
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: f0265e47cc86f73eac914d6c42610d8a
Run description: Rel feedback BM25 D1 T5 Thres 200

UrsinusPwrA¶

Run ID: UrsinusPwrA
Participant: Ursinus_College
Track: Legal
Year: 2008
Submission: 8/4/2008
Type: automatic
Task: adhoc
MD5: 692a6ca61802433bb0c306ea67ac8fb8
Run description: Power norm with 2006 optimal values

UrsinusPwrB¶

Run ID: UrsinusPwrB
Participant: Ursinus_College
Track: Legal
Year: 2008
Submission: 8/5/2008
Type: automatic
Task: adhoc
MD5: 732f0665c44cc3f1640a07f002f79580
Run description: Automated relevance feedback 1 Doc 5 Terms, threshold 1.0. Power Normalization.

UrsinusPwrC¶

Run ID: UrsinusPwrC
Participant: Ursinus_College
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: e487e57d0bd1356f4b98f78bb62d508d
Run description: rel feedback power norm top 3 docs, top 5 terms, thres 1.4

UrsinusVa¶

Run ID: UrsinusVa
Participant: Ursinus_College
Track: Legal
Year: 2008
Submission: 8/6/2008
Type: automatic
Task: adhoc
MD5: 89bb1cacfdee5f23e5a19d265af9f889
Run description: Vector baseline for comparison with EDLSI runs

uvabase¶

Run ID: uvabase
Participant: UAmsterdam
Track: Legal
Year: 2008
Submission: 7/31/2008
Type: automatic
Task: adhoc
MD5: e420b08355ca7d63d70dd9e5307ae4bd
Run description: Base Run - Took just the text of RequestText, with a regular stoplist. K and Kh are by our algorithm.

wat1fuse¶

Run ID: wat1fuse
Participant: UWIR
Track: Legal
Year: 2008
Submission: 7/31/2008
Type: automatic
Task: adhoc
MD5: eb6b975a8bd57b25af340e448bde181f
Run description: A fusion of the following 8 runs - boolean - relaxed boolean - okapi on request text field - okapi on request text field, stemming on - okapi using the text in the boolean - 4-gram okapi on request text field - 4-gram okapi using the text in the boolean - 4-gram okapi using words in request text field K was produce using an average of a log and non-log linear regression on scores. Kh is half of K

wat2text¶

Run ID: wat2text
Participant: UWIR
Track: Legal
Year: 2008
Submission: 7/31/2008
Type: automatic
Task: adhoc
MD5: 790b5b911477b31a9fbde30196d7bde3
Run description: A fusion of the following 4 runs - okapi on request text field - okapi on request text field, stemming on - 4-gram okapi on request text field - 4-gram okapi using words in request text field K =25000 Kh is half of K

wat3nobool¶

Run ID: wat3nobool
Participant: UWIR
Track: Legal
Year: 2008
Submission: 7/31/2008
Type: automatic
Task: adhoc
MD5: f362e10df6a492ce2fc5becce5b66395
Run description: wat3nobool A fusion of the following 8 runs excluding documents retrieved by the reference boolean - boolean - relaxed boolean - okapi on request text field - okapi on request text field, stemming on - okapi using the text in the boolean - 4-gram okapi on request text field - 4-gram okapi using the text in the boolean - 4-gram okapi using words in request text field K =100000 Kh is half of K

wat4fuse¶

Run ID: wat4fuse
Participant: UWIR
Track: Legal
Year: 2008
Submission: 7/31/2008
Type: automatic
Task: adhoc
MD5: c8606ff08d205f2abbb4f2335849aa90
Run description: A fusion of the following 8 runs - boolean - relaxed boolean - okapi on request text field - okapi on request text field, stemming on - okapi using the text in the boolean - 4-gram okapi on request text field - 4-gram okapi using the text in the boolean - 4-gram okapi using words in request text field K was produce using a linear regression on scores. Kh is half of K

wat5fuse¶

Run ID: wat5fuse
Participant: UWIR
Track: Legal
Year: 2008
Submission: 7/31/2008
Type: automatic
Task: adhoc
MD5: 068560d9f7247403a364ff466b3f1d18
Run description: A fusion of the following 8 runs - boolean - relaxed boolean - okapi on request text field - okapi on request text field, stemming on - okapi using the text in the boolean - 4-gram okapi on request text field - 4-gram okapi using the text in the boolean - 4-gram okapi using words in request text field K was produce using a log linear regression on scores. Kh is half of K

wat6fuse¶

Run ID: wat6fuse
Participant: UWIR
Track: Legal
Year: 2008
Submission: 7/31/2008
Type: automatic
Task: adhoc
MD5: e44065330eced3c7f20449a35428dd37
Run description: A fusion of the following 8 runs - boolean - relaxed boolean - okapi on request text field - okapi on request text field, stemming on - okapi using the text in the boolean - 4-gram okapi on request text field - 4-gram okapi using the text in the boolean - 4-gram okapi using words in request text field K = 25000 Kh is half of K

wat7fuse¶

Run ID: wat7fuse
Participant: UWIR
Track: Legal
Year: 2008
Submission: 7/31/2008
Type: automatic
Task: adhoc
MD5: bfe00d86efdc057f29cc0cf000ad0962
Run description: A fusion of the following 8 runs - boolean - relaxed boolean - okapi on request text field - okapi on request text field, stemming on - okapi using the text in the boolean - 4-gram okapi on request text field - 4-gram okapi using the text in the boolean - 4-gram okapi using words in request text field K = 100000 Kh is half of K

wat8fuse¶

Run ID: wat8fuse
Participant: UWIR
Track: Legal
Year: 2008
Submission: 7/31/2008
Type: automatic
Task: adhoc
MD5: a819b9044028bc77cf64eb87505ba6a8
Run description: A fusion of the following 8 runs - boolean - relaxed boolean - okapi on request text field - okapi on request text field, stemming on - okapi using the text in the boolean - 4-gram okapi on request text field - 4-gram okapi using the text in the boolean - 4-gram okapi using words in request text field K = B Kh is half of K

xb¶

Run ID: xb
Participant: UAmsterdam
Track: Legal
Year: 2008
Submission: 7/31/2008
Type: automatic
Task: adhoc
MD5: 2d5c9bab8d86c8c2a304b57f2b155673
Run description: Extended stoplist - with K constant as B and Kh as B/2

xconst¶

Run ID: xconst
Participant: UAmsterdam
Track: Legal
Year: 2008
Submission: 7/31/2008
Type: automatic
Task: adhoc
MD5: e5dde18fec2d3ff2567dba4b15f13c35
Run description: Extended stoplist - with K constant as 16904 and Kh constant as k*0.5

xk¶

Run ID: xk
Participant: UAmsterdam
Track: Legal
Year: 2008
Submission: 7/31/2008
Type: automatic
Task: adhoc
MD5: 2b0bc686faae6cfbdfd4d303ac0bb0ca
Run description: We took the request text, parsed by an extended stop list, based on the topics of 2006 and 2007. We used our algorithm for K and Kh.