DataScience@BI seminar with Nick Koning

DataScience@BI seminar invites Nick Koning, Assistant professor, Erasmus University Rotterdam to give a talk within the field of econometrics.

Abstract

An unfortunate feature of traditional hypothesis testing is the necessity to pre-specify a significance level α to bound the ‘size’ of the test: its probability to falsely reject the hypothesis. Indeed, a data-dependent selection of α would generally distort the size, possibly making it larger than the selected level α. We develop post-hoc α hypothesis testing, which guarantees that there is no such size distortion in expectation, even if the level α is arbitrarily selected based on the data. Unlike regular p-values, resulting ‘post-hoc p-values’ allow us to ‘reject at level p’ and still provide this guarantee. Moreover, they can easily be combined since the product of independent post-hoc p-values is also a post-hoc p-value. Interestingly, we find that p is a post-hoc p-value if and only if 1/p is an e-value, a recently introduced measure of evidence. This reveals what e-values truly are in the context of a hypothesis testing problem. Post-hoc α hypothesis testing eliminates the need for standardized levels such as α = 0.05, which takes away incentives for p-hacking and contributes to solving the file-drawer problem.

Key research areas
Foundational topics in statistics e-values Testing exchangeability and group invariance.

• Read more about the speaker
• Visit our website to read more about the Department of Data Science and Analytics