Forster, Malcolm R. (2000): "Key Concepts in Model
Selection: Performance and Generalizability" Journal of Mathematical Psychology,
in the same volume.
Note: You need Adobe Acrobat Reader 3.0,
or later, to read and print this article.
is the same as the published version. (285
What is model selection? What are the goals of model selection? What are the
methods of model selection, and how do they work? Which methods perform better than
others, and in what circumstances? These questions rest on a number of key concepts in a
relatively underdeveloped field. The aim of this essay is to explain some background
concepts, highlight some of the results in this special issue, and to add my own.
The standard methods of model selection include classical
hypothesis testing, maximum likelihood, Bayes method, minimum description length,
cross-validation and Akaikes information criterion. They all provide an
implementation of Occams razor, in which parsimony or simplicity is balanced against
goodness-of-fit. These methods primarily take account of the sampling errors in parameter
estimation, although their relative success at this task depends on the circumstances.
However, the aim of model selection should also include the ability of a model to
generalize to predictions in a different domain. Errors of extrapolation, or
generalization, are different from errors of parameter estimation. So, it seems that
simplicity and parsimony may be an additional factor in managing these errors, in which
case the standard methods of model selection are incomplete implementations of