This page was last edited on 05/04/02 by Malcolm R Forster

Forster, Malcolm R. (2000):  "Key Concepts in Model Selection: Performance and Generalizability" Journal of Mathematical Psychology, 44, 205-231.

Other papers in the same volume.

   PDF files

Note: You need Adobe Acrobat Reader 3.0, or later, to read and print this article.

This version is the same as the published version.  (285 KB)

What is model selection? What are the goals of model selection? What are the methods of model selection, and how do they work? Which methods perform better than others, and in what circumstances? These questions rest on a number of key concepts in a relatively underdeveloped field. The aim of this essay is to explain some background concepts, highlight some of the results in this special issue, and to add my own.
     The standard methods of model selection include classical hypothesis testing, maximum likelihood, Bayes method, minimum description length, cross-validation and Akaike’s information criterion. They all provide an implementation of Occam’s razor, in which parsimony or simplicity is balanced against goodness-of-fit. These methods primarily take account of the sampling errors in parameter estimation, although their relative success at this task depends on the circumstances. However, the aim of model selection should also include the ability of a model to generalize to predictions in a different domain. Errors of extrapolation, or generalization, are different from errors of parameter estimation. So, it seems that simplicity and parsimony may be an additional factor in managing these errors, in which case the standard methods of model selection are incomplete implementations of Occam’s razor.