The New Science of Simplicity

This page was last edited on 02/07/02 by Malcolm R Forster

  PDF file

Note: You need Adobe Acrobat Reader 3.0, or later.

Published version  (292 KB)

Publication Data



Forster, Malcolm R. (2001): "The New Science of Simplicity," in Arnold Zellner, Hugo Keuzenkamp, and Michael McAleer (eds.), Simplicity, Inference and Modelling, pp. 83-117.  Cambridge: Cambridge University Press.

 

Table of Contents

 

  1. The problem

  2. Preliminaries

  3. A milieu of methods and an easy example

  4. Predictive accuracy as a goal of model selection

  5. A ‘normality’ assumption and the geometry of parameter space

  6. Comparing selection criteria

  7. The charge that AIC is inconsistent

  8. Summary of results

 

Abstract

There was time when statistics was a mere footnote to the methodology of science; concerned only with the mundane task of estimating the size of observational errors and designing experiments. That was because statistical methods assumed a fixed background "model", and only methodology was concerned with the selection of the model. Simplicity was an issue in methodology, but not in statistics. All that has changed. Statistics has expanded to cover model selection, and simplicity has appeared in statistics with a form and precision that it never attained in the methodology of science. This is the new science of simplicity.
   This paper lays a foundation for all forms of model selection from hypothesis testing and cross validation to the newer AIC and BIC methods that trade off simplicity and fit. These methods are evaluated with respect to a common goal of maximizing predictive accuracy. Within this framework, there is no relevant sense in which AIC is inconsistent, despite an almost universally cited claim to the contrary. Asymptotic properties are not pivotal in the comparison of selection methods. The real differences show up in intermediate sized data sets. Computer computations suggest that there are no global optimums—the dilemma is between performing poorly in one set of circumstance or performing poorly in another.