Thomas A. Romberg (1994). Classroom instruction that fosters mathematical thinking and problem solving: connections between theory and practice. In Alan H. Schoenfeld (Ed.) (1994). Mathematical thinking and problem solving. Erlbaum. questia

An experimenter tells you she has a rule in mind about triples of whole numbers. She also tells you that the triple '2 4 6' conforms to that rule. Your task is to discover the rule by thinking of new triples, the experimenter will tell you whether a new triplet conforms to the rule or not.

P. C. Wason (1960). On the failure to eliminate hypotheses in a conceptual task. Quarterly Journal of Experimental Psychology, 12, 129-140. The original article does not seem to be available on the www. Try cuil.com, search: wason problem. Note: it is not the same as the Wason selection task, another famous problem in the same line of research.

Jonathan St. B. T. Evans (1989). Bias in human reasoning: Causes and consequences. Erlbaum. See p. 45 ff. This book is not available in questia.com. However, searching questia on its title resuts in a number of related books that are.

• abstract The modeling theory of instruction is widely applied in high-school instruction, and seldom in university physics. One reason is lack of familiarity with models in the physics classroom. Recent developments in modeling theory could inspire a new interest in application of modeling theory at the university level. This paper follows one class through a modeling cycle, including development, application and adaptation, extension, and revision in an effort to reify the role of models in a class. The benefits of modeling instruction are identified and include effective knowledge organization, consistency with accepted scientific practice, and improved problem solving through use of multiple representations.
• I have yet to make a detailed study of this line of instructional development. It is part of the research using the Force Concept Inventory: David Hestenes, Malcolm Wells, and Gregg Swackhamer (1992). Force Concept Inventory. The Physics Teacher, Vol. 30, 141-158. pdf.
• Eric Thomas Brewe (2002). Inclusion of the energy thread in the introductory physics curriculum: An example of long-term conceptual and thematic coherence. A Dissertation Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy pdf

Prove that every number greater than 4 is the sum of two odd primes.

George Pólya (1954/68). Mathematics and plausible reasoning. Volume I: Induction and analogy in mathematics. Volume II: Patterns of plausible inference. Princeton University Press. pp. 4-7.

7.4 History

The Cambridge mathematical tripos in the nineteenth century is an extremely competitive examination spread out over some eight full days, sometimes in bitter cold as well. There are a few publications giving the full examinations, as well as the intended answers. Almost every problem is for almost every examinee a difficult one, a very difficult one. It is possible to see for yourself, as some of these oldies have been reprinted in the Elibron Classic series.

Exact numbers on the difficulty of the problems are extremely scarce, but two series of marks of the best 100 honours students were published by James Galton in his book 'Hereditary genius,' see here. Galton also explains the examination procedure. Reproduced here, using the original source code:

7.5 Literature

more literature

Many items are as yet on my 'to do' list: they are mentioned here, but not used in the above text yet.

David McMath, Marianna Rozenfeld, and Richard Sommer (www accessed 2006). A Computer Environment for Writing Ordinary Mathematical Proofs. (Education Program for Gifted Youth, Stanford University) pdf

• Abstract The EPGY Theorem-Proving Environment is designed to help students write ordinary mathematical proofs. The system, used in a selection of computer-based proof-intensive mathematics courses, allows students to easily input mathematical expressions, apply proof strategies, verify logical inference, and apply mathematical rules. Each course has its own language, database of theorems, and mathematical inference rules. The main goal of the project is to create a system that imitates standard mathematical practice in the sense that it allows for natural modes of reasoning to generate proofs that look much like ordinary textbook proofs. Additionally, the system can be applied to an unlimited number of proof exercises
• "The history of the students' actions are recorded so that we can Òplay backÓ a student's proof from start to finish, displaying all steps including those that were deleted by the student. Using this feature, we have had the opportunity to examine many students' proofs. The analysis of these proofs has influenced both the development of the system and our presentation of the material in our courses."

Hanne ten Berge, Stephan Ramaekers en Albert Pilot (2004). The design of authentic tasks that promote higher-order learning. Paper presented at the EARLI-SIG Higher Education/IKIT-conference, June 18-21, 2004. pdf

• abstract This study focuses on the use of authentic cases to provoke higher-order learning. It is based on a review of recent research literature and focuses in particular on the design features of these cases. First, we looked into the characteristics of real life problems academics are confronted with in their professional practices and the way they solve those problems. In the transformation of these to educational tasks, elements of authenticity get lost. Awareness of the features that are crucial to authenticity of the case seems important if higher-order learning is to be achieved. The literature not only provided indications for the design of authentic cases, we also came across examples of negative effects of authentic cases and improper design. The results of our search show that the impact of authenticity on learning outcomes is promising but it does not present conclusive results. Authenticity appears to be a much too wide ranging formulation to be helpful in the design of case attributes. In the future we will focus more closely on the coherence of the factors that play a part in higher-order learning.

Bruce G. Buchanan and Richard O. Duda (1982). Principles of rule-based expert systems. Heuristic Programming Projec Report no. HPP-82-14. pdf

• Bruce G. Buchanan (1982). Partial bibliography of work on expet systems. HPP-82-30 pdf
• R. O. DUDA and J. G. GASCHNIG (1981). Knowledge-based expert systems come of age. BYTE, 238-284.
  • De computer geprogrammeerd als expert probleemoplosser: Stand van zaken; vooral medische diagnostiek: de praktische resultaten die Kunstmatige-Intelligentie-onderzoek ook oplevert.
• Brent M. Dingle (2001). What AI may (or may not) be. http://students.cs.tamu.edu/dingle/Papers/What%20AI%20may.pdf [Dead link? May 3, 2009]
• Fred D. Fagg, III, and Peter D. Bergsman (1997). Computer-implemented decision management system with dynamically generated questions and answer choices. United States Patent 5978784 html
  • abstract A computer-based method for assisting a user in making decisions in the process of completing a task is shown and described. The method of the system includes providing a set of questions for completing the task, asking the user a first question from the set, and providing a choice of answers to the first question. The system also provides advice from its knowledge base for deciding which of the answers to the first question to select. The choice of answer, however, remains under the control of the user, who can exercise judgment based on the user's expertise and the advice provided by the decision management system. The method continues by asking following questions, with the following questions, answers and advice influenced by the user's previous answers. Throughout the decision-making process, the user retains control of the answers if so desired, with the system providing advice from its knowledge base. The system is particularly suited for document assembly in allowing a professional to determine what provisions are suitable for a document to be built from information and judgments provided by the professional and the knowledge base of the system. The system also includes an authoring program for preparing applications that run on the system. Using an intuitive program block approach of the authoring program, an author can construct an application by placing questions, answer choices, advice and textual and logical provisions in named program blocks. The author then assembles and arranges these blocks to build a desired application program.
• Paul T. Baffes, Siddarth Subramanian and Shane V. Nugent (2001). System and method for dynamic knowledge generation and distribution. United States Patent http://164.195.100.11/netacgi/nph-Parser?Sect2=PTO1&Sect2=HITOFF&p=1&u=%2Fnetahtml%2Fsearch-bool.html&r=1&f=G&l=50&d=PALL&RefSrch=yes&Query=PN%2F6292792 [Dead link? May 3, 2009]
• Albertus Laing Jordaan (2004). Design and implementation of a supervisory expert system for hot rolling process optimisation. Dissertation submitted in the fulfilment of the requirements for the degree master of Engineering, in the Faculty of Engineering at the Rand Afrikaans University. http://0-etd.uj.ac.za.raulib.rau.ac.za/theses/available/etd-11302004-101528/restricted/SUPERVISORYEXPERTSYSTEM20040818Final.pdf [Dead link? May 3, 2009]
  • Met actuele literatuurlijst over expert systems en fuzzy logics
• D. Sleeman, Haym Hirsh, Ian Ellery and In-Yung Kim (1990). Extending Domain Theories: Two Case Studies in Student Modeling. Machine Learning, 5, 11-37. [Ik heb dit artikel zelf nog niet kunnen zien, lijkt me interessant, zie het abstract]
  • abstract By its very nature, artificial intelligence is concerned with investigating topics that are ill-defined and ill-understood. This paper describes two approaches to expanding a good but incomplete theory of a domain. The first uses the domain theory as far as possible and fills in specific gaps in the reasoning process, generalizing the suggested missing steps and adding them to the domain theory. The second takes existing operators of the domain theory and applies perturbations to form new plausible operators for the theory. The specific domain to which these techniques have been applied is high-school algebra problems. The domain theory is represented as operators corresponding to algebraic manipulations, and the problem of expanding the domain theory becomes one of discovering new algebraic operators. The general framework used is one of generate and testÑgenerating new operators for the domain and using tests to filter out unreasonable ones. The paper compares two algorithms, INFER and MALGEN, examining their performance on actual data collected in two Scottish schools and concluding with a critical discussion of the two methods.

Michelene T. H. Chi, Robert Glaser, and Marshall J. Farr (Eds) (1989). The nature of expertise. Hillsdale: Erlbaum. (ao: Beth Adelson and Elliot Soloway: A model of software design - Jeanette A. Lawrence: Expertise on the bench: Modeling magistrates' judicional decision-making - James F. Voss and Timothy A. Post: On the solving of ill-structured problems - Guy J. Groen and Vimla L. Patel: The relationship between comprehension and reasoning in medical expertise - Alan Lesgold, Harriet Rubinson, Paul Feltovich, Robert Glaser, Dale Klopfer, and Yen Wang: Expertise in complex skill: diagnosing X-ray pictures - William J. Clancey: Acquiring, representing, and evaluating a competence model of diagnostic strategy)

A. S. Elstein, L. S. Shulman en S. A. Sprafkan (1978). Medical problem solving: An analysis of clinial reasoning. Harvard University Press.

• [Moet ik nog opzoeken. Lambert Schuwirth verwijst ernaar]

Peter Gerjets, Katharina Scheiter and Richard Catrambone (2004). Designing Instructional Examples to Reduce Intrinsic Cognitive Load: Molar versus Modular Presentation of Solution Procedures. Instructional Science 32, 33-58,

• abstract It is usually assumed that successful problem solving in knowledge-rich domains depends on the availability of abstract problem-type schemas whose acquisition can be supported by presenting students with worked examples. Conventionally designed worked examples often focus on information that is related to the main components of problemtype schemas, namely on information related to problem-category membership, structural task features, and category-specific solution procedures. However, studying these examples might be cognitively demanding because it requires learners to simultaneously hold active a substantial amount of information in working memory. In our research, we try to reduce intrinsic cognitive load in example-based learning by shifting the level of presenting and explaining solution procedures from a Ômolar' view - that focuses on problem categories and their associated overall solution procedures - to a more Ômodular' view where complex solutions are broken down into smaller meaningful solution elements that can be conveyed separately. We review findings from five of our own studies that yield evidence for the fact that processing modular examples is associated with a lower degree of intrinsic cognitive load and thus, improves learning.

Peter Gerjets, Richard Catrambone and Katharina Scheiter (2006). Reducing Cognitive Load and Fostering Cognitive Skill Acquisition: Benefits of Category-Avoiding Instructional Examples. http://www.ccm.ua.edu/pdfs/99.pdf [Dead link? May 3, 2009]

• abstract In this paper, we provide evidence against the common idea that worked examples should be designed to convey problem categories and category-specific solution procedures. Instead we propose that instructional examples should be designed in a way that supports the understanding of relations between structural problem features and individual solution steps, i.e. relations that hold below the category level. We illustrate in the domain of probability word problems how category-avoiding instructional examples can be constructed. In two experiments we provide evidence that category-avoiding examples reduce cognitive load during learning and that they foster subsequent problem-solving performance.

Adriaan D. Groot (1978). Thought and choice in chess. Den Haag: Mouton, 1978.

Guershon Harel and Larry Sowder (1998). Students' proof schemes. Research on Collegiate Mathematics Education, Vol. III. In E. Dubinsky, A. Schoenfeld, & J. Kaput (Eds.), AMS, 234-283. pdf

Guershon Harel and Larry Sowder (In Press). Toward a comprehensive perspective on proof, In F. Lester (Ed.), Second Handbook of Research on Mathematics Teaching and Learning, National Council of Teachers of Mathematics. http://math.ucsd.edu/~harel/downloadablepapers/TCPOLTOP.pdf [Dead link? May 3, 2009]

Ross Honsberger (1978). Mathematical morsels. The Mathematical Association of America.

• "This is not a book of problems which are posed for you to solve (...), but as a showcase for some of mathematics' minor miracles."
• "Mathematics abounds in bright ideas. No matter how long and hard one pursues her, mathematics never seems to run out of exciting surprises. And by no means are these gems to be found only in diffcult work at an advanced level. All kinds of simple notions are full of ingenuity."
• "The present volume discusses scores of elementary problems which have been culled primarily from the American Mathematical Monthly, 1894-1975."
• As a non-mathematician you should know that mathematicians like to be members of mathematical clubs. One of the things these mathematicians do in their club is put problems and solve problems. There must be a library of books assembling these problems. It is a culture, the pinnacle of which might be found in Cambridge in the context of the Mathematical Tripos examinations (17th - 20th century). The phenomenon should be a warning to mathematical course and test builders; all of this is highly artificial, and predicated on the unique interests of mathematicians themselves, not of the pupils they teach, or of other professionals needing their mathematical results.
• The 91 gems in the book might be used to as materials to discuss and experience the doing of mathematics, not in the manner of solving puzzles and problems, but trying to understand what it is that is called mathematical thinking, and do some of it yourself. This is not discussed by Honsberger, by the way, you will have to modify the material yourself to make it fit for classroom use.

David H. Jonassen (1997). Instructional design model for well-structured and ill-structured problem-solving learning outcomes. Educational Technology: Research and Development, 45, 65-95.

• Jonassen, D.H. (2004). Learning to solve problem: An instructional design guide. San Francisco, CA: Jossey-Bass. [I have not seen the book. Having seen a range of articles by Jonassen, I am pretty sure the book sums up what has been published already elsewhere. The publisher's site offers an excerpt of the text: pdf

D. Jonassen (2000). Toward a Design Theory of Problem Solving. Educational Technology Research and Development, 2000, 48, 63-85. http://www.coe.missouri.edu/~jonassen/PSPaper%20final.pdf [Dead link? May 3, 2009]

• abstract Problem solving is generally regarded as the most important cognitive activity in everyday and professional contexts. Most people are required and rewarded for solving problems. However, learning to solve problems is too seldom required in formal educational settings, in part, because our understanding of its processes is limited. Instructional-design research and theory has devoted too little attention to the study of problem-solving processes or methods and models for supporting problem-solving learning. In this article, I describe differences among problems in terms of their structuredness, complexity, and domain specificity (abstractness). Then, I briefly describe a variety of individual differences (factors internal to the problem solver) that affect problem solving. Finally, I articulate a typology of problems, each type of which engages different cognitive, affective, and conative processes and therefore necessitates different instructional support. The purpose of this paper is to propose a meta-theory of problem solving in order to initiate dialogue andresearch rather than offering a definitive answer regarding its processes.

David H. Jonassen (2003). Using cognitive tools to represent problems. Journal of Research in Technology in Education, 35, 362-381. http://tiger.coe.missouri.edu/~jonassen/ProbRep.zip [Dead link? May 3, 2009]

• abstract The premise of this paper is that the key to problem solving is adequately representing the problem to be solved. Most research has focused on how problems are (re)presented to learners. The assumption that those external representations naturally map onto learners' internal representations of problems has not been confirmed. New research has examined the role of tools for externalizing learners' internal representations. Descriptions of how three kinds of cognitive toolsÑsemantic networks, expert systems, and systems modeling toolsÑcan be used to externalize learner's internal representations are provided. Research is needed to study the efficacy of these tools for supporting problem solving.

David H. Jonassen (2004?). The future of learning: Learning to solve problems. pdf

David H. Jonassen and Woei Hung (under review). Learning to Troubleshoot: A New Theory-Based Design Architecture. pdf

• from the abstract The architecture includes three essential components: a multi-layered conceptual model of the system that includes topographic, function, strategic, and procedural representations; a simulator that requires the learner to generate hypotheses, reconcile the hypotheses to the system mode, test the hypotheses, and interpret the results from the test; and a case library that uses a case-based reasoning engine to access relevant stories of troubleshooting experiences as advice for the learner. This novel architecture can be used to develop learning environments for different kinds of troubleshooting.

Ton de Jong (Ed.) (2003). Knowledge management Interactive Training System. KITS consortium. pdf

J. H. Larkin, J. H. (1981). Enriching formal knowledge: a model for learning to solve textbook physics problems. In J. R. Anderson Cognitive skills and their acquisition. Hillsdale, New Jersey, Erlbaum, 1981.

Peter Lehman (1996). Will that be on the exam? Schema theory and testing in sociology. Teaching Sociology, October. pdf

• Geeft uitvoerig uitgewerkt voorbeeld. Leuk artikel, zeker voor sociologen.

Lesh, R., & Harel, G. (2003). Problem solving, modeling, and local conceptual development. International Journal of Mathematics Thinking and Learning, 5, 157-189. http://math.ucsd.edu/~harel/downloadablepapers/Problem%20solving,%20modeling,%20and%20local%20conceptual%20development.pdf [ead link? May 3, 2009]

Alan Newell and Herbert A. Simon (1972). Human problem solving. Englewood Cliffs, New Jersey, Prentice Hall, 1972.

• Een standaardwerk over het oplossen van problemen. Staat met beide voeten in de Kunstmatige-Intelligentietraditie, probeert een beschrijvende (psychologische) theorie van het oplossen van problemen te geven.

G. Polya (1957/1971). How to solve it. A new aspect of mathematical method. Princeton, New Jersey, Princeton University Press.

• Een 'klassiek' boek over het aanpakken van problemen, vooral wiskundige problemen.

Joseph Psotka, L. Dan Massey and Sharon A. Mutter. (Ed.) (1988). Intelligent tutoring systems. Lessons learned. Hillsdale/NJ: Lawrence Erlbaum. 552 pp., isbn 0805801928 paperback (a.o.: ) (contents) (http://www.cs.cmu.edu/~mazda/ITS/literature.htm Textbooks on intelligent tutoring systems)

L. W. T. Schuwirth, D.E. Blackmore , E. Mom , F. van den Wildenberg , H.E.J.H. Stoffers en C.P.M. van der Vleuten (1999). How to write short cases for assessing problem-solving skills. Medical Teacher, 21, 144-150.

• summary In assessment of problem solving the use of short case-based testing is a promising development. In this approach an examination consists of large numbers of short cases each of which contain a small number of questions.These questions are aimed at essential decisions.Writing such cases, however, is not easy. In this article a description of this type of examination is provided.Also strategies and pitfalls are described in writing these cases. These strategies pertain to the selection of essential decisions, the careful writing of cases and questions and the selection of question formats.
• L. W. T. Schuwirth (1998). An approach to the assessment of medical problem solving: computerised case-based testing. Maastricht: Datawyse Publications. (Thesis University of Maastricht)
• Lambert Schuwirth (2006). Toetsen met korte casussen. In Henk van Berkel en Anneke Bax: Toetsen in het hoger onderwijs (p. 127-143). Houten: Bohn Stafleu van Loghum.
• John Norcini (2004). Back to the Future: Clinical Vignettes and the Measurement of Physician Performance. [Editorial] Annals of Internal Medicine, 141 #10, 16 november pdf
• J B Battles, S L Wilkinson and S J Lee (2004). Using standardised patients in an objective structured clinical examination as a patient safety tool. Qual. Saf. Health Care,13, 46-50. pdf

Daniel L. Schwartz and Taylor Martin (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22, 129-184. as paper

• Offers extensive treatment of several problems, solution approaches. Assessment philosophy: assessment should be instructive itself. Educational level: grade 9.

Herbert A. Simon (1980). Problem solving and education. In D. T. Tuma and F. Reif (1980).

Herbert A. Simon (1973). The structure of ill-structured problems. Artificial Intelligence, 4, 181-201. Reprinted in Herbert A. Simon (1977). Models of discovery and other topics in the methods of science. Dordrecht: Reidel.

• Kees Dorst (2006). Design problems and design paradoxes. Design issues, 22 #3 pdf
• Armand Hatchuel (2002). Towards Design Theory and expandable rationality: The unfinished program of Herbert Simon. To be published in the Journal of Management and Governance 5 :3-4 2002. pdf

L. Sowder and G. Harel (2003). Case Studies of Mathematics Majors' Proof Understanding, Production, and Appreciation. Canadian Journal of Science, Mathematics and Technology Education. 3, 251-26. http://math.ucsd.edu/~harel/downloadablepapers/CJSMTE.pdf [Dead link? May 3, 2009]

D. T. Tuma and F. Reif (Eds). (1980). Problem solving and education: issues in teaching and research. Hillsdale, New Jersey, Erlbaum.

Renée E. Weiss (2003). Designing Problems to Promote Higher-Order Thinking. New Directions for Teaching and Learning 95, 25-31. pdf

• "A crucial aspect of problem-based learning (PBL) is the actual design of the problem to be solved"

W. A. Wickelgren (1974). How to solve problems, Elements of a theory of problems and problem solving, San Francisco, Freeman, 1974.

• Benadert de aanpak van problemen vooral vanuit de verworvenheden op dit gebied van Kunstmatige Intelligentie.

Links

Software

"Soar is a general cognitive architecture for developing systems that exhibit intelligent behavior. Researchers all over the world are using Soar."

• Allen Newell (1990). Unified theories of cognition. Cambridge, Mass.: Harvard University Press. Dit boek is het begin van Soar.
• Richard L. Lewis (1999 draft/2001). Cognitive theory, SOAR. pdf
• Richard L. Lewis (1999). Cognitive modeling, symbolic. n Wilson, R. and Keil, F. (eds.), The MIT Encyclopedia of the Cognitive Sciences. Cambridge, MA: MIT Press. pdf
• Manual 8 pdf
• A list of publications using Soar here

IHMC Cmap Tools site

• Open source software for the construction of concept maps

• List of publications using concept mapping html

Weisstein, Eric W. "Archimedes' Cattle Problem." From MathWorld--A Wolfram Web Resource. http://mathworld.wolfram.com/ArchimedesCattleProblem.html