Tim Stelzer and Gary Gladding (2001). The Evolution of WebBased Activities in Physics at Illinois. Forum on Education, Fall. html
an impossibility
Prove that every number greater than 4 is the sum of two odd primes.
The conjecture is known as
Goldbach's conjecture, no one has been able to prove it. It is, of course, easy to construct lots and lots of examples showing the conjecture to be true in those particular cases: 8 = 3 + 5; 10 = 3 + 7 = 5 + 5; etcetera. This kind of problem is very special indeed. Fermat's Last Theorem used to be another example, it has now been proved by Andrew Wiles and Richard Taylor
Wiki.
George Pólya (1954/68). Mathematics and plausible reasoning. Volume I: Induction and analogy in mathematics. Volume II: Patterns of plausible inference. Princeton University Press. pp. 47.
7.4 History
The Cambridge mathematical tripos in the nineteenth century is an extremely competitive examination spread out over some eight full days, sometimes in bitter cold as well. There are a few publications giving the full examinations, as well as the intended answers. Almost every problem is for almost every examinee a difficult one, a very difficult one. It is possible to see for yourself, as some of these oldies have been reprinted in the Elibron Classic series.
William Walton, Charles Mackenzie. Solutions of the Problems and Riders Proposed in the SenateHouse Examination for 1854. By the Moderators and Examiners. With an Appendix, Containing the Examination Papers in Full.
Elibron Classics, 2002, 238 pages.
ISBN 140216131X paperback
ISBN 1402128185 hardcover
Replica of 1854 edition by Macmillan and Co., Cambridge.
Exact numbers on the difficulty of the problems are extremely scarce, but two series of marks of the best 100 honours students were published by James Galton in his book 'Hereditary genius,' see here. Galton also explains the examination procedure. Reproduced here, using the original source code:
19
Scale of merit among the men who obtain mathematical honours at Cambridge.
The results of two years are thrown into a single table.
The total number of marks obtainable in each year was 17,000.
Number of marks obtained by
candidates.
Number of candidates in the two
years, taken together, who obtained
those marks.
Under 500
24¹
500 to 1,000
74
1,000 to 1,500
38
1,500 to 2,000
21
2,000 to 2,500
11
2,500 to 3,000
8
3,000 to 3,500
11
3,500 to 4,000
5
4,000 to 4,500
2
4,500 to 5,000
1
5,000 to 5,500
3
5,500 to 6,000
1
6,000 to 6,500
0
6,500 to 7,000
0
7,000 to 7,500
0
7,500 to 8,000
1
200
The precise number of marks obtained by the senior wrangler in the more
remarkable of these two years was 7,634; by the second wrangler in the
same year, 4,123; and by the lowest man in the list of honours, only 237.
Consequently, the senior wrangler obtained nearly twice as many marks as
the second wrangler, and more than thirtytwo times as many as the lowest
man. I have received from another examiner the marks of a year in which
the senior wrangler was conspicuously eminent.
1
I have included in this table only the first 100 men in each year. The omitted residue is too
small to be important. I have omitted it lest, if the precise numbers of honour men were stated,
those numbers would have served to identify the years. For reasons already given, I desire to
afford no data to serve that purpose.

7.5 Literature
more literature
Many items are as yet on my 'to do' list: they are mentioned here, but not used in the above text yet.
David McMath, Marianna Rozenfeld, and Richard Sommer (www accessed 2006). A Computer Environment for Writing Ordinary Mathematical Proofs. (Education Program for Gifted Youth, Stanford University) pdf
 Abstract The EPGY TheoremProving Environment is designed to help students write ordinary mathematical proofs. The system, used in a selection of computerbased proofintensive mathematics courses, allows students to easily input mathematical expressions, apply proof strategies, verify logical inference, and apply mathematical rules. Each course has its own language, database of theorems, and mathematical inference rules. The main goal of the project is to create a system that imitates standard mathematical practice in the sense that it allows for natural modes of reasoning to generate proofs that look much like ordinary textbook proofs. Additionally, the system can be applied to an unlimited number of proof exercises
 "The history of the students' actions are recorded so that we can Òplay backÓ a student's proof from start to finish, displaying all steps including those that were deleted by the student. Using this feature, we have had the opportunity to examine many students' proofs. The analysis of these proofs has influenced both the development of the system and our presentation of the material in our courses."
Hanne ten Berge, Stephan Ramaekers en Albert Pilot (2004). The design of authentic tasks that promote higherorder learning. Paper presented at the EARLISIG Higher Education/IKITconference, June 1821, 2004. pdf
 abstract This study focuses on the use of authentic cases to provoke higherorder learning. It is based on a review of recent research literature and focuses in particular on the design features of these cases. First, we looked into the characteristics of real life problems academics are confronted with in their professional practices and the way they solve those problems. In the transformation of these to educational tasks, elements of authenticity get lost. Awareness of the features that are crucial to authenticity of the case seems important if higherorder learning is to be achieved. The literature not only provided indications for the design of authentic cases, we also came across examples of negative effects of authentic cases and improper design. The results of our search show that the impact of authenticity on learning outcomes is promising but it does not present conclusive results. Authenticity appears to be a much too wide ranging formulation to be helpful in the design of case attributes. In the future we will focus more closely on the coherence of the factors that play a part in higherorder learning.
Bruce G. Buchanan and Richard O. Duda (1982). Principles of rulebased expert systems. Heuristic Programming Projec Report no. HPP8214. pdf
 Bruce G. Buchanan (1982). Partial bibliography of work on expet systems. HPP8230 pdf
 R. O. DUDA and J. G. GASCHNIG (1981). Knowledgebased expert systems come of age. BYTE, 238284.
 De computer geprogrammeerd als expert probleemoplosser: Stand van zaken; vooral medische diagnostiek: de praktische resultaten die KunstmatigeIntelligentieonderzoek ook oplevert.
 Brent M. Dingle (2001). What AI may (or may not) be. http://students.cs.tamu.edu/dingle/Papers/What%20AI%20may.pdf [Dead link? May 3, 2009]
 Fred D. Fagg, III, and Peter D. Bergsman (1997). Computerimplemented decision management system with dynamically generated questions and answer choices. United States Patent 5978784 html
 abstract A computerbased method for assisting a user in making decisions in the process of completing a task is shown and described. The method of the system includes providing a set of questions for completing the task, asking the user a first question from the set, and providing a choice of answers to the first question. The system also provides advice from its knowledge base for deciding which of the answers to the first question to select. The choice of answer, however, remains under the control of the user, who can exercise judgment based on the user's expertise and the advice provided by the decision management system. The method continues by asking following questions, with the following questions, answers and advice influenced by the user's previous answers. Throughout the decisionmaking process, the user retains control of the answers if so desired, with the system providing advice from its knowledge base. The system is particularly suited for document assembly in allowing a professional to determine what provisions are suitable for a document to be built from information and judgments provided by the professional and the knowledge base of the system. The system also includes an authoring program for preparing applications that run on the system. Using an intuitive program block approach of the authoring program, an author can construct an application by placing questions, answer choices, advice and textual and logical provisions in named program blocks. The author then assembles and arranges these blocks to build a desired application program.
 Paul T. Baffes, Siddarth Subramanian and Shane V. Nugent (2001). System and method for dynamic knowledge generation and distribution. United States Patent http://164.195.100.11/netacgi/nphParser?Sect2=PTO1&Sect2=HITOFF&p=1&u=%2Fnetahtml%2Fsearchbool.html&r=1&f=G&l=50&d=PALL&RefSrch=yes&Query=PN%2F6292792 [Dead link? May 3, 2009]
 Albertus Laing Jordaan (2004). Design and implementation of a supervisory expert system for hot rolling process optimisation. Dissertation submitted in the fulfilment of the requirements for the degree master of Engineering, in the Faculty of Engineering at the Rand Afrikaans University. http://0etd.uj.ac.za.raulib.rau.ac.za/theses/available/etd11302004101528/restricted/SUPERVISORYEXPERTSYSTEM20040818Final.pdf [Dead link? May 3, 2009]
 Met actuele literatuurlijst over expert systems en fuzzy logics
 D. Sleeman, Haym Hirsh, Ian Ellery and InYung Kim (1990). Extending Domain Theories: Two Case Studies in Student Modeling. Machine Learning, 5, 1137. [Ik heb dit artikel zelf nog niet kunnen zien, lijkt me interessant, zie het abstract]
 abstract By its very nature, artificial intelligence is concerned with investigating topics that are illdefined and illunderstood. This paper describes two approaches to expanding a good but incomplete theory of a domain. The first uses the domain theory as far as possible and fills in specific gaps in the reasoning process, generalizing the suggested missing steps and adding them to the domain theory. The second takes existing operators of the domain theory and applies perturbations to form new plausible operators for the theory. The specific domain to which these techniques have been applied is highschool algebra problems. The domain theory is represented as operators corresponding to algebraic manipulations, and the problem of expanding the domain theory becomes one of discovering new algebraic operators. The general framework used is one of generate and testÑgenerating new operators for the domain and using tests to filter out unreasonable ones. The paper compares two algorithms, INFER and MALGEN, examining their performance on actual data collected in two Scottish schools and concluding with a critical discussion of the two methods.
Michelene T. H. Chi, Robert Glaser, and Marshall J. Farr (Eds) (1989). The nature of expertise. Hillsdale: Erlbaum. (ao: Beth Adelson and Elliot Soloway: A model of software design  Jeanette A. Lawrence: Expertise on the bench: Modeling magistrates' judicional decisionmaking  James F. Voss and Timothy A. Post: On the solving of illstructured problems  Guy J. Groen and Vimla L. Patel: The relationship between comprehension and reasoning in medical expertise  Alan Lesgold, Harriet Rubinson, Paul Feltovich, Robert Glaser, Dale Klopfer, and Yen Wang: Expertise in complex skill: diagnosing Xray pictures  William J. Clancey: Acquiring, representing, and evaluating a competence model of diagnostic strategy)
A. S. Elstein, L. S. Shulman en S. A. Sprafkan (1978). Medical problem solving: An analysis of clinial reasoning. Harvard University Press.
 [Moet ik nog opzoeken. Lambert Schuwirth verwijst ernaar]
Peter Gerjets, Katharina Scheiter and Richard Catrambone (2004). Designing Instructional Examples to Reduce Intrinsic Cognitive Load: Molar versus Modular Presentation of Solution Procedures. Instructional Science 32, 3358,
 abstract It is usually assumed that successful problem solving in knowledgerich domains depends on the availability of abstract problemtype schemas whose acquisition can be supported by presenting students with worked examples. Conventionally designed worked examples often focus on information that is related to the main components of problemtype schemas, namely on information related to problemcategory membership, structural task features, and categoryspecific solution procedures. However, studying these examples might be cognitively demanding because it requires learners to simultaneously hold active a substantial amount of information in working memory. In our research, we try to reduce intrinsic cognitive load in examplebased learning by shifting the level of presenting and explaining solution procedures from a Ômolar' view  that focuses on problem categories and their associated overall solution procedures  to a more Ômodular' view where complex solutions are broken down into smaller meaningful solution elements that can be conveyed separately. We review findings from five of our own studies that yield evidence for the fact that processing modular examples is associated with a lower degree of intrinsic cognitive load and thus, improves learning.
Peter Gerjets, Richard Catrambone and Katharina Scheiter (2006). Reducing Cognitive Load and Fostering Cognitive Skill Acquisition: Benefits of CategoryAvoiding Instructional Examples. http://www.ccm.ua.edu/pdfs/99.pdf [Dead link? May 3, 2009]
 abstract In this paper, we provide evidence against the common idea that worked examples should be designed to convey problem categories and categoryspecific solution procedures. Instead we propose that instructional examples should be designed in a way that supports the understanding of relations between structural problem features and individual solution steps, i.e. relations that hold below the category level. We illustrate in the domain of probability word problems how categoryavoiding instructional examples can be constructed. In two experiments we provide evidence that categoryavoiding examples reduce cognitive load during learning and that they foster subsequent problemsolving performance.
Adriaan D. Groot (1978). Thought and choice in chess. Den Haag: Mouton, 1978.
Guershon Harel and Larry Sowder (1998). Students' proof schemes. Research on Collegiate Mathematics Education, Vol. III. In E. Dubinsky, A. Schoenfeld, & J. Kaput (Eds.), AMS, 234283. pdf
Guershon Harel and Larry Sowder (In Press). Toward a comprehensive perspective on proof, In F. Lester (Ed.), Second Handbook of Research on Mathematics Teaching and Learning, National Council of Teachers of Mathematics. http://math.ucsd.edu/~harel/downloadablepapers/TCPOLTOP.pdf [Dead link? May 3, 2009]
Ross Honsberger (1978). Mathematical morsels. The Mathematical Association of America.
 "This is not a book of problems which are posed for you to solve (...), but as a showcase for some of mathematics' minor miracles."
 "Mathematics abounds in bright ideas. No matter how long and hard one pursues her, mathematics never seems to run out of exciting surprises. And by no means are these gems to be found only in diffcult work at an advanced level. All kinds of simple notions are full of ingenuity."
 "The present volume discusses scores of elementary problems which have been culled primarily from the American Mathematical Monthly, 18941975."
 As a nonmathematician you should know that mathematicians like to be members of mathematical clubs. One of the things these mathematicians do in their club is put problems and solve problems. There must be a library of books assembling these problems. It is a culture, the pinnacle of which might be found in Cambridge in the context of the Mathematical Tripos examinations (17th  20th century). The phenomenon should be a warning to mathematical course and test builders; all of this is highly artificial, and predicated on the unique interests of mathematicians themselves, not of the pupils they teach, or of other professionals needing their mathematical results.

The 91 gems in the book might be used to as materials to discuss and experience the doing of mathematics, not in the manner of solving puzzles and problems, but trying to understand what it is that is called mathematical thinking, and do some of it yourself. This is not discussed by Honsberger, by the way, you will have to modify the material yourself to make it fit for classroom use.
David H. Jonassen (1997). Instructional design model for wellstructured and illstructured problemsolving learning outcomes. Educational Technology: Research and Development, 45, 6595.
 Jonassen, D.H. (2004). Learning to solve problem: An instructional design guide. San Francisco, CA: JosseyBass. [I have not seen the book. Having seen a range of articles by Jonassen, I am pretty sure the book sums up what has been published already elsewhere. The publisher's site offers an excerpt of the text: pdf
D. Jonassen (2000). Toward a Design Theory of Problem Solving. Educational Technology Research and Development, 2000, 48, 6385. http://www.coe.missouri.edu/~jonassen/PSPaper%20final.pdf [Dead link? May 3, 2009]
 abstract Problem solving is generally regarded as the most important cognitive activity in everyday and professional contexts. Most people are required and rewarded for solving problems. However, learning to solve problems is too seldom required in formal educational settings, in part, because our understanding of its processes is limited. Instructionaldesign research and theory has devoted too little attention to the study of problemsolving processes or methods and models for supporting problemsolving learning. In this article, I describe differences among problems in terms of their structuredness, complexity, and domain specificity (abstractness). Then, I briefly describe a variety of individual differences (factors internal to the problem solver) that affect problem solving. Finally, I articulate a typology of problems, each type of which engages different cognitive, affective, and conative processes and therefore necessitates different instructional support. The purpose of this paper is to propose a metatheory of problem solving in order to initiate dialogue andresearch rather than offering a definitive answer regarding its processes.
David H. Jonassen (2003). Using cognitive tools to represent problems. Journal of Research in Technology in Education, 35, 362381. http://tiger.coe.missouri.edu/~jonassen/ProbRep.zip [Dead link? May 3, 2009]
 abstract The premise of this paper is that the key to problem solving is adequately representing the problem to be solved. Most research has focused on how problems are (re)presented to learners. The assumption that those external representations naturally map onto learners' internal representations of problems has not been confirmed. New research has examined the role of tools for externalizing learners' internal representations. Descriptions of how three kinds of cognitive toolsÑsemantic networks, expert systems, and systems modeling toolsÑcan be used to externalize learner's internal representations are provided. Research is needed to study the efficacy of these tools for supporting problem solving.
David H. Jonassen (2004?). The future of learning: Learning to solve problems. pdf
David H. Jonassen and Woei Hung (under review). Learning to Troubleshoot: A New TheoryBased Design Architecture. pdf
 from the abstract The architecture includes three essential components: a multilayered conceptual model of the system that includes topographic, function, strategic, and procedural representations; a simulator that requires the learner to generate hypotheses, reconcile the hypotheses to the system mode, test the hypotheses, and interpret the results from the test; and a case library that uses a casebased reasoning engine to access relevant stories of troubleshooting experiences as advice for the learner. This novel architecture can be used to develop learning environments for different kinds of troubleshooting.
Ton de Jong (Ed.) (2003). Knowledge management Interactive Training System. KITS consortium. pdf
J. H. Larkin, J. H. (1981). Enriching formal knowledge: a model for learning to solve textbook physics problems. In J. R. Anderson Cognitive skills and their acquisition. Hillsdale, New Jersey, Erlbaum, 1981.
Peter Lehman (1996). Will that be on the exam? Schema theory and testing in sociology. Teaching Sociology, October. pdf
 Geeft uitvoerig uitgewerkt voorbeeld. Leuk artikel, zeker voor sociologen.
Lesh, R., & Harel, G. (2003). Problem solving, modeling, and local conceptual development. International Journal of Mathematics Thinking and Learning, 5, 157189. http://math.ucsd.edu/~harel/downloadablepapers/Problem%20solving,%20modeling,%20and%20local%20conceptual%20development.pdf [ead link? May 3, 2009]
Alan Newell and Herbert A. Simon (1972). Human problem solving. Englewood Cliffs, New Jersey, Prentice Hall, 1972.

Een standaardwerk over het oplossen van problemen. Staat met beide voeten in de KunstmatigeIntelligentietraditie, probeert een beschrijvende (psychologische) theorie van het oplossen van problemen te geven.
G. Polya (1957/1971). How to solve it. A new aspect of mathematical method. Princeton, New Jersey, Princeton University Press.

Een 'klassiek' boek over het aanpakken van problemen, vooral wiskundige problemen.
Joseph Psotka, L. Dan Massey and Sharon A. Mutter. (Ed.) (1988). Intelligent tutoring systems. Lessons learned. Hillsdale/NJ: Lawrence Erlbaum. 552 pp., isbn 0805801928 paperback (a.o.: ) (contents) (http://www.cs.cmu.edu/~mazda/ITS/literature.htm Textbooks on intelligent tutoring systems)
L. W. T. Schuwirth, D.E. Blackmore , E. Mom , F. van den Wildenberg ,
H.E.J.H. Stoffers en C.P.M. van der Vleuten (1999). How to write short cases for assessing problemsolving skills. Medical Teacher, 21, 144150.
 summary In assessment of problem solving the use of short casebased testing is a promising development. In this approach an examination consists of large numbers of short cases each of which contain a small number of questions.These questions are aimed at essential decisions.Writing such cases, however, is not easy. In this article a description of this type of examination is provided.Also strategies and pitfalls are described in writing these cases. These strategies pertain to the selection of essential decisions, the careful writing of cases and questions and the selection
of question formats.

L. W. T. Schuwirth (1998). An approach to the assessment of medical problem solving: computerised casebased testing. Maastricht: Datawyse Publications. (Thesis University of Maastricht)

Lambert Schuwirth (2006). Toetsen met korte casussen. In Henk van Berkel en Anneke Bax: Toetsen in het hoger onderwijs (p. 127143). Houten: Bohn Stafleu van Loghum.

John Norcini (2004). Back to the Future: Clinical Vignettes and the Measurement of
Physician Performance. [Editorial] Annals of Internal Medicine, 141 #10, 16 november pdf

J B Battles, S L Wilkinson and S J Lee (2004). Using standardised patients in an objective structured clinical examination as a patient safety tool. Qual. Saf. Health Care,13, 4650. pdf
Daniel L. Schwartz and Taylor Martin (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22, 129184. as paper
 Offers extensive treatment of several problems, solution approaches. Assessment philosophy: assessment should be instructive itself. Educational level: grade 9.
Herbert A. Simon (1980). Problem solving and education. In D. T. Tuma and F. Reif (1980).
Herbert A. Simon (1973). The structure of illstructured problems. Artificial Intelligence, 4, 181201. Reprinted in Herbert A. Simon (1977). Models of discovery and other topics in the methods of science. Dordrecht: Reidel.
 Kees Dorst (2006). Design problems and design paradoxes. Design issues, 22 #3 pdf
 Armand Hatchuel (2002). Towards Design Theory and expandable rationality: The unfinished program of Herbert Simon. To be published in the Journal of Management and Governance 5 :34 2002. pdf
L. Sowder and G. Harel (2003). Case Studies of Mathematics Majors' Proof Understanding, Production, and Appreciation. Canadian Journal of Science, Mathematics and Technology Education. 3, 25126. http://math.ucsd.edu/~harel/downloadablepapers/CJSMTE.pdf [Dead link? May 3, 2009]
D. T. Tuma and F. Reif (Eds). (1980). Problem solving and education: issues in teaching and research. Hillsdale, New Jersey, Erlbaum.
Renée E. Weiss (2003). Designing Problems to Promote HigherOrder Thinking. New Directions for Teaching and Learning 95, 2531. pdf
 "A crucial aspect of problembased learning (PBL) is the actual design of the
problem to be solved"
W. A. Wickelgren (1974). How to solve problems, Elements of a theory of problems and problem solving, San Francisco, Freeman, 1974.

Benadert de aanpak van problemen vooral vanuit de verworvenheden op dit gebied van Kunstmatige Intelligentie.
Links
Software
"Soar is a general cognitive architecture for developing systems that exhibit intelligent behavior. Researchers all over the world are using Soar."

Allen Newell (1990). Unified theories of cognition. Cambridge, Mass.: Harvard University Press. Dit boek is het begin van Soar.

Richard L. Lewis (1999 draft/2001). Cognitive theory, SOAR. pdf

Richard L. Lewis (1999). Cognitive modeling, symbolic. n Wilson, R. and Keil, F. (eds.), The MIT Encyclopedia of the Cognitive Sciences. Cambridge, MA: MIT Press. pdf

Manual 8 pdf

A list of publications using Soar here
IHMC Cmap Tools site
Weisstein, Eric W. "Archimedes' Cattle Problem." From MathWorldA Wolfram Web Resource. http://mathworld.wolfram.com/ArchimedesCattleProblem.html
http://www.benwilbrink.nl/projecten/06examples7.htm