Overfitting and optimism

In Chapter 5, we discuss causes of overfitting in predictive regression models, and its consequences, specifically optimism in performance estimates (Table 5.1). These issues are presented in a Practical Exercise, which starts with re-creating Fig 5.2 in R, and a critical look at Fig 5.3 and 5.4. Next, we discuss stepwise selection in the presence of pure noise predictors, in line with a paper by Mundry and Nunn. Finally, we perform bootstrap resampling to study optimism.