The learning curve is used to project expected utilities in the future. The applet does just that: plot the expected utility for every episode and for every bar within it. Remember excepted utility? Is was evaluated in The Predictor, the predictive test score distribution. In fact, in orde to be able to plot the curve of expected utilities, for every one of them the predictive test score distribution has to be generated (either by simulating or by analysing it). The following applet has been built to generate just one such projected predictive score distribution, at the highest episode declared in the menu.
The reported expected utilities should be the same (analysis) or approximately the same (simulation) as those reported in the first applet, given the same parameter values in the menu.
[Gigerenzer c.s. chapter one on expecting uncertain outcomes in the sixteenth and seventeenth centuries]
The interpretation of the expected utility curves is that the point of inflection in the curve, if it does have one, indicates a good strategic point to aim for. The verdict on good or better strategies cannot be final yet, however, because somehow the cost of time yet to be invested has to be reckoned with. The case where the expected utility curve does not have a point of inflection illustrates the problem: if investing more time always results in still better expected utility, then when should one stop to invest time? The next module presents a solution to the problem.
The menus have by now become quite numerous, by now all of them must be familiar. Otherwise, look back at the Generator or at the Learning module.
The crux in the construction of the curves of expected utility is the translation of the likelihood of mastery forward as well as backward along the learning path. The translation is deterministic, as explained in the treatment of the learning model. For every point of mastery specified in ordr to evaluate the likelihood function, its projection in time must be specified. For every point its functional value will remain the same, however, unles the simulation procedure is used. The problem with this straightforward procedure is that it will result in strange scrambles of transformed points of mastery. The solution to this problem is to follow a reverse procedure: from regular grids of mastery for every future of past likelihood to be evaluated, find for every mastery value its translated 'original' mastery value at the moment of preliminary testing, and evaluate the likelihood belonging to it. The mathematics involved use the inverse of the learning function. The fact that the inverse of the learning curve is needed is a drawback in the procedure. However, it should be possible to use a graphical procedure to find the inverse of the learning function that has been specified, thus preserving the generalness of the model.
Click the code to get a readable picture.
Stratified sampling from two subdomains
The algorithm for the stratified sampling case roughly consists of evaluating the projected predictive distributions for each domain separately, and only then combining the predictions to construct the predictive distrution for the combined score.
Using the same learning model in both subdomains, an alternative method could be to construct the likelihood on the combined score, and evaluating the projection based on that combined likelihood. In fact, the learning parameters would have to be equal also, in which case the whole exercise would be trivial.
The applet currently does not offer the opportunity to specify different kinds of models for the two subdomains (in the future it might be implemented), but evidently the learning model specified will in general have different parameter values in the two subdomains.
expected utility functions: simulation-analysis fit
The figure shows a modest simulation and the corresponding analytical curves of expected utility.
projected prediction: simulation-analysis fit
The first step in testing the applet for the projected predictive distribution is to compare the analytical and simulation results. The plot is the result of a 100.000 observations simulation. [note: this is module 6a, not 5a]
projected prediction: simulation-analysis fit: the stratified case
Stratified subdomain sampling has been impelmented, but has not yet been tested. The picture shows a simulation-analysis plot; click on it to get it in the original format.
The two applets in this chapter share a number of methods, but otherwise are quite different.
Cronbach, Lee J. , and Goldine C. Gleser (1957/1965). Psychological tests and personnel decisions. Urbana, Illiois: University of Illinois Press.
[Establishes the foundations of a decision-theoretic approach to decisions in the psychological and educational domain. Gives a correct treatment of the concept of expected utility functions. The concept of expected utility functions is related to that of aptitude-treatment interactions, see Lee J. Cronbach and Richard E. Snow (1977). Aptitudes and instructional methods. New York: Irvington.]
Gigerenzer, Gerd, Zeno Swijtink, Theodore Porter, Lorraine Daston, John Beatty, and Lorenz Krüger (1989). The empire of chance. How probability changed science and everyday life. Cambridge: Cambridge University Press.
Wilbrink, Ben (1980). Optimale kriterium gerefereerde grensskores zijn eenvoudig te vinden. Tijdschrift voor Onderwijsresearch, 5, 49-62. [56k html + 3 gifjes]
Wilbrink, Ben (1980). Enkele radicale oplossingen voor kriterium gerefereerde grensskores. Tijdschrift voor Onderwijsresearch, 5, 112-125. [44k html + 80k gif]
The twin publications in the Tijdschrift voor Onderwijsresearch explicate the misconception about expected utility in the contemporary literature on criterion-refenced testing. English abstracts, self-explaining pictures.
[expected utility functions in the literature on decision making, Cronbach and Gleser, etcetera.
The psychometric literature using decision theory in the seventies and later, that is disconnected from the body of science mentioned above]
For the applet itself click spa_applets.htm#6,
Mail your opinion, suggestions, critique, experience on/with the SPA
http://www.benwilbrink.nl/projecten/spa_expectation.htm