 |
2. Probability in Rating Systems
|
|
|
|
The basic assumption of the Elo System is that the chess performance of an individual player is a random variable that can be described by the normal curve [E1, 1.31]. But what exactly is varying in this random variable?
As applied to a single game, performance is an abstraction which cannot be measured objectively. It consists of all the judgments, decisions, and action of the contestant in the course of the game. Perhaps a panel of experts in the art of the game could evaluate each move on some arbitrary scale and crudely express the total performance numerically, even as is done in boxing and gymnastics [E1, 1.32].
[2.1] P(W, L, D) = [ N! / (W! L! D!) ] . P(win)W . P(loss)L . P(draw)D if we know the probabilities P(win), P(loss), and P(draw). Let us consider the outcomes for ten games (N = 10), where P(win) = .5, P(loss) = .2, and P(draw) = .3, and let us express the results in terms of points scored. Since [2.1] must be calculated for 66 three-way partitions of 10, this is best done with a computer program (Downloads). The results are as follows:
Although Elo presented [2.1] almost as an afterthought, it provides a convincing demonstration of the "first and basic assumption" of his system [E2, Part 1, "Form Varies"]. In the development of a rating theory, however, it is a dead end. Variation of performance thus described yields no useful definition of probability for rating theory, which is primarily concerned to relate percentage scores to ratings. The crucial percentage score is the mean value .65, which becomes increasingly prominent as sample size increases. The position taken here is that this long-term value is the link to probability theory. Elo's rather nebulous definition of performance leads to the central concept of his system: the Percentage Expectancy Curve, which is patterned on a well-known probability function, either the normal or the logistic. The Percentage Expectancy Curve relates percentage score to rating difference. Precisely how it does this is a crucial question for the system. We are shown two overlapping distributions [E1, 8.23] and are told that the shaded portion of one "represents the probability that the lower rated player will outperform the higher." Apparently, if a player's performance is greater than that of the opponent, he/she wins; if the performance is lower, he/she loses. This argument, aside from the objection that it leaves draws completely out of account, hangs on a tenuous concept of performance. Outperforming the opponent seems equivalent in every respect to winning; yet this would suggest a binomial distribution, if not trinomial. For Elo the Percentage Expectancy Curve
was patently a probability function, and he could make no sense of the
objection that it was not. To avoid semantic arguments, the
objection is better taken as a distinction of terms. Since a percentage score
may be thought of as an estimate of probability, defined as a long-term
percentage, there is reason enough to regard the Percentage Expectancy
Curve as a function that relates probability to rating difference.
In this broad sense, it is a probability function. A more precise
use of the term is restricted to those functions that arise
in probability theory from a mathematical analysis of variability, such as
the normal curve or the logistic, and these we may call true
probability functions. A function that merely maps probability to
another variable without some justification based on variability analysis
would thus be called an arbitrary probability function. Such
functions include those based arbitrarily on a true probability function,
which is the case of the Percentage Expectancy Curve. If the
Percentage Expectancy Curve itself were a true probability function, it
would be derived independently from distributions of rating difference,
however these might arise, but these can hardly be known without a
pre-established definition of ratings.
|
