3 Experimental Methodology

Chapter 3
Experimental Methodology

In the previous chapter we have looked at the assumptions, models, and related work underpinning this thesis. In this chapter we introduce the experimental methodology generally employed in IR and also adopted in this thesis. We start by discussing the notion of relevance, detailing standard forms of evaluation, and significance testing. We then describe the data sets that will be used in later chapters. We conclude with a section discussing retrieval model parameters.

3.1 Relevance
3.2 Evaluation
  3.2.1 Evaluation Measures
  3.2.2 Statistical Significance Testing
3.3 Test Collections
  3.3.1 TREC Robust 2004
  3.3.2 TREC Terabyte 2004–2006
  3.3.3 TREC Relevance Feedback 2008
  3.3.4 TREC Web 2009
  3.3.5 CLEF Domain-Specific 2007–2008
  3.3.6 TREC Genomics 2004–2006
3.4 Parameter Settings
3.5 Summary

PrevTail

Front

Chapter 3Experimental Methodology

Chapter 3
Experimental Methodology