p-values for model evaluation

In the analysis of experimental results it is often necessary to pass a judgment on the validity of a model as a representation of the data. A quantitative procedure to decide whether a model provides a good description of data is often based on a specific test statistic and a p-value summarizing both the data and the statistic's sampling distribution. Although there is considerable confusion concerning the meaning of p-values, leading to their misuse, they are nevertheless of practical importance in common data analysis tasks. We motivate the application of p-values using a Bayesian argumentation. We then describe commonly and less commonly known test statistics and how they are used to define p-values. The distribution of these are then extracted for examples modeled on typical new physics searches in high energy physics. We comment on their usefulness for determining goodness-of-fit and highlight some common pitfalls.