This page was last edited on 02/07/02 by Malcolm R Forster

Work in progress. 

  PDF files

Note: You need Adobe Acrobat Reader 3.0, or later, to read and print this article.  It is free.

Version 1 is a one and a half-spaced PDF file. Last updated 12/13/98.  50 KB.

Two kinds of error commonly arise in statistical inference. The most common one is sampling error, arising from small samples. The second is the error arising from unrepresentative samples. Such errors occur in curve-fitting examples when the curves are fitted in one domain and used for prediction in another. We refer to this error as the extrapolation error. The problem with extrapolation is that standard model selection methods, such as classical hypothesis testing, AIC, BIC, do not correct for extrapolation error. Nevertheless, the simple method of judging models by their success in prediction is shown to perform better in one simulated example. This note formulates the following question in a precise mathematical terms: When does this method work? There is no precise answer to this question at the present time, even in quite simple contexts.

The resolution of a DILEMMA: Learning algorithms are reliable if and only if nature is uniform. Does this explain why some learning algorithms are reliable?  The objection is that the "uniformity of nature" is either left unspecified or is defined as "whatever ensures the reliability of induction", which is unhelpful. Extrapolation Error offers a mathematical formulation of the uniformity of nature that is neither vague nor blatantly circular.