Ben Wilbrink literatuur: onderwijsonderzoek

The right to know what is supposed to improve. What problem is being solved? For example, when I’ve been to schools or districts implementing a one-to-one tablet/laptop policy, I’ve always asked what it’s meant to do. The modal response is a blank look followed by the phrase “we don’t want our kids left behind.” Behind in what? In what way are kids elsewhere with devices zooming ahead?
The right to know the means by which improvement will be measured. How will we know things are getting better? If you’re trying to improve students’ understanding of math, for example, are you confident that you have a metric that captures that construct? Are you sure scores on that metric will be comparable in the future to those you’re looking at now? How big an increase will be deemed a success?
The right to know the approximate time by which this improvement is expected. A commitment to an intervention shouldn’t be open-ended. At some point we must evaluate how it’s going.
The right to know what will be done if the goal is or is not met. Naturally, conditions may change, but let’s have a plan. If we don’t meet our target, will we quit? Keep trying for a while? Tweak it?
The right to know what evidence exists that the intervention will work as expected. Is the evidence from actual classrooms or is it laboratory science (plus some guesswork)? If classrooms, were they like ours? In how many classrooms was it tried?
The right to have your experience and expertise acknowledged. If the intervention sounds to you and your colleagues like it cannot work, this issue should be addressed in detail, not waved away with the phrase “all the research supports it.” The fact that it sounds fishy to experienced people doesn’t mean it can’t work, but whoever is pitching it should have a deep enough understanding of the mechanisms behind the intervention to be able to say why it sounds fishy, and why that’s not a problem.

Mooi overzicht. Via Dylan Wiliam 20-12-2013

De conclusie is dat, hoewel het onderzoek beperkingen kent, verbeteringen mogelijk zijn in de wijze waarop de inspectie de leeropbrengsten van scholen beoordeelt.

Although a substantial body of literature indicates that Head Start has meaningful effects on children’s cognitive and achievement outcomes, there are a variety of factors that might explain the magnitude of Head Start effects that are about the evaluation research designs, not just the program itself. Thus, comparing results from evaluation studies requires attention to both how the studies are conducted and which instruments are used to assess outcomes.

This article examines the impacts of summer instruction and test-based grade retention in New York City.

The author considers studies of the impact of intended versus experienced instructional regimes. Both are important; however, intended regimes are well measured and accessible to randomized trials, whereas experienced instruction is measured with error and not amenable to randomization. Multiyear sequences of experienced instruction are of central interest but pose special methodological challenges. A 2-year study of intensive mathematics instruction illustrates these ideas.

Jencks and others (1972) have shown how tight the distribution of school achievement scores is after controlling for nonschool background factors. Our various regressions support that finding. (...) The Michigan data provide evidence of unusually effective schools. (...) For the New York City data the results were mixed. (...) The project Talent data showed no evidence of consistently overachieving schools apart from that predicted by chance alone. This negative finding seems even stronger when one considers that only SES was used as a regressor.

In a sample of 7262 college freshmen attending 24 colleges and universities, the student scores for several academic achievement tests, the student scores for several scales of extracurricular achievement, and the student average school grades were intercorrelated. The correlations between these measures of academic and non-academic accomplishments are generally neglegible (median r = .04). The results can be attributed neither to a narrow range of academic talent nor to non-linear relationships. The results strongly suggest that academic and non-academic accomplishment are relatively independent dimensions of talent. The implications of the findings for the selection of talented persons and the conservation of talent were discussed.

Reflections on schools and adolescents, James S. Coleman; youth and adolesence - a historical and cultural perspective, Torsten Husen; Coleman's contribution to understanding youth and adolescence, Denise B. Kandel; private schooling in post-communist Poland, Barbara Heyns; early schooling and educational inequality - socioeconomic disparities in children's learning, Karl L. Alexander and Doris R. Entwistle; Coleman's contributions to education - theory, research styles and empirical research, James J. Heckman and Derek Neal; the sociological contribution to social policy research, Martin Bulmer; the political context of social policy research, Sally B. Kilgore; games with simulated environments, Sarane Spence Boocock; methodological individualism and collective behaviour, Benjamin Zablocki; mobility measurement revisited, David J. Bartholomew; self-employment and entrepreneurship - a study of entry and exit, Peter Abell; educational opportunities and school effects, Aage B. Sorensen; the violation of normative rules and the issue of rationality in individual judgements, Michael Imbar; foundational problems in theoretical sociology, Thomas J. Fararo; rational schoice as grand theory - James Coleman's normative contribution to social theory, Adrian Favell; constitutionalism versus relationalism - two versions of rational choice sociology, Siegwart Lindenberg; analyzing the economy - on the contribution of James S. Coleman, Richard Swedberg; can rational action theory unify future social science?, Randall Collins; a vision for sociology, James S. Coleman.

i.a. Pamela Sammons: Ethical issues and statistical work - Alison Kelly: Education or indoctrination? The ethics of school based action research - Helen Simons: Ethics of case study in educational research and evaluation - Harry Torrance: Ethics and politics in the study of assessment researchgate.net the whole book

Leerlingen in de tweede fase krijgen maar weinig studievaardigheden bijgebracht. En de hoeveelheid huiswerk is gering. Het is de vraag of deze leerlingen wel het beste uit zichzelf willen en kunnen halen.

. . . describes the education systems as ones that have promised justice, inclusion and equity for all in formal policy but in practice actually fail to do so. A turn in recent years to market politics as a means of appeasement is given attention and critiqued for having worsened the situation.

References Adams, G. (1996). Project Follow Through: In-depth and beyond. Effective School Practices, 15, 43–56. Google Scholar Adams, G., Engelmann, S. (1996). Research on Direct Instruction: 25 years beyond DISTAR. Seattle, WA: Educational Achievement Systems. Google Scholar American Institutes for Research. (1999). An educator’s guide to schoolwide reform. Arlington, VA: Author. Google Scholar Barbash, S. (2012). Clear teaching: With Direct Instruction, Siegfried Engelmann discovered a better way of teaching. Arlington, VA: Education Consumers Foundation. Google Scholar Becker, W. C., Gersten, R. (1982). A follow-up of Follow Through: The later effects of the Direct Instruction model on children in fifth and sixth grades. American Educational Research Journal, 19, 75–92. Google Scholar, Link Benner, G. J., Nelson, J. R., Stage, S. A., Ralston, N. C. (2010). The influence of fidelity of implementation on the reading outcomes of middle school students experiencing reading difficulties. Remedial and Special Education, 32, 79–88. Google Scholar, Link Bereiter, C., Engelmann, S. (1966). Teaching disadvantaged children in the preschool. Englewood Cliffs, NJ: Prentice Hall. Google Scholar Bereiter, C., Kurland, M. (1996). A constructive look at Follow Through results. Effective School Practices, 15, 17–32. Google Scholar Bloom, H. S., Hill, C. J., Black, A. B., Lipsey, M. W. (2008). Performance trajectories and performance gaps as achievement effect-size benchmarks for educational interventions. Journal of Research on Educational Effectiveness, 1, 289–328. Google Scholar, Crossref Borenstein, M., Hedges, L. V., Higgins, J. P. T., Rothstein, H. R. (2009). Introduction to meta-analysis. New York, NY: Wiley. Google Scholar, Crossref Borman, G. D., Hewes, G. M., Overman, L. T., Brown, S. (2003). Comprehensive school reform and achievement: A meta-analysis. Review of Educational Research, 73, 125–230. Google Scholar, Link Carlson, C. D., Francis, D. J. (2002). Increasing the reading achievement of at-risk children through Direct Instruction: Evaluation of the Rodeo Institute for Teacher Excellence (RITE). Journal of Education for Students Placed at Risk, 7, 141–166. Google Scholar, Crossref Collins, M., Carnine, D. (1988). Evaluating the field test revision process by comparing two versions of a reasoning skills CAI program. Journal of Learning Disabilities, 21, 375–379. Google Scholar, Link Cook, T. D., Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Chicago, IL: Rand McNally. Google Scholar Coughlin, C. (2011). Research on the effectiveness of Direct Instruction (NIFDI Technical Report 2011-4). Eugene, OR: National Institute for Direct Instruction Google Scholar Coughlin, C. (2014). Outcomes of Engelmann’s Direct Instruction: Research syntheses. In Stockard, J. (Ed.), The science and success of Engelmann’s Direct Instruction (pp. 25–54). Eugene, OR: NIFDI Press. Google Scholar Cohen, J. (1977). Statistical Power Analysis for the Behavioral Sciences (Rev. ed.). New York, NY: Academic Press. Google Scholar Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Google Scholar Dedrick, R. F., Ferron, J. M., Hess, M. R., Hogarty, K. Y., Kromrey, J. D., Lang, T. R., . . . Lee, R. S. (2009). Multilevel modeling: A review of methodological issues and applications. Review of Educational Research, 79, 69–102. Google Scholar, Link, ISI Engelmann, S. (1999). Theory of mastery and acceleration. In Lloyd, J. W., Kame’enui, E. J., Chard, D. (Eds.), Issues in educating students and disabilities (pp. 177–195). Mahwah, NJ: Lawrence Erlbaum. Google Scholar Engelmann, S. (2007). Teaching needy kids in our backward system: 42 years of trying. Eugene, OR: ADI Press. Google Scholar Engelmann, S. (2014a). The dreaded standards. In Wood, T. W. (Ed.), Engelmann’s Direct Instruction: Selected writings from the past half century (pp. 414–421). Eugene, OR: NIFDI Press. Google Scholar Engelmann, S. (2014b). Research from the inside: The development and testing of DI programs. In Stockard, J. (Ed.), The science and success of Engelmann’s Direct Instruction (pp. 3–24). Eugene, OR: NIFDI Press. Google Scholar Engelmann, S. (2014c). Successful and confident students with Direct Instruction. Eugene, OR: NIFDI Press. Google Scholar Engelmann, S., Becker, W. C., Carnine, D., Gersten, R. (1988). The Direct Instruction Follow Through model: Design and outcomes. Education and Treatment of Children, 11, 303–317. Google Scholar Engelmann, S., Carnine, D. (1991). Theory of instruction: Principles and applications (Rev. ed.) Eugene, OR: ADI Press. (Originally published 1982) Google Scholar Engelmann, S., Carnine, D. (2011). Could John Stuart Mill have saved our schools? Verona, WI: Full Court Press. Google Scholar Engelmann, S., Colvin, G. (2006). Rubric for identifying authentic Direct Instruction programs. Eugene, OR: Engelmann Foundation. Google Scholar Engelmann, S., Steely, D. (2004). Inferred functions of performance and learning. Mahwah, NJ: Lawrence Erlbaum. Google Scholar Gersten, R. M., Carnine, D. W., Williams, P. B. (1982). Measuring implementation of a structured educational model in an urban school district: An observational approach. Educational Evaluation and Policy Analysis, 4, 67–79. Google Scholar, Link Gersten, R., Carnine, D., Zoref, L., Cronin, D. (1986). A multifaceted study of change in seven inner-city schools. The Elementary School Journal 86, 257–276. Google Scholar, Crossref Grossen, B. (1996). The story behind Project Follow Through. Effective School Practices 15(1), 4–9. Google Scholar Hattie, J. A. C. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. New York, NY: Routledge. Google Scholar Huitt, W. G., Monetti, D. M., Hummel, J. H. (2009). Direct approach to instruction. In Reigeluth, C., Carr-Chellman, A. (Eds.), Instructional-design theories and models: Vol. 3. Building a common knowledge base (pp. 73–98). Mahwah, NJ: Lawrence Erlbaum. Google Scholar Kalaian, H., Raudenbush, S. W. (1996). A multivariate mixed linear model for meta-analysis. Psychological Methods, 1, 227–235. Google Scholar, Crossref Kennedy, M. M. (1978). Findings from the Follow Through planned variation study. Educational Researcher, 7(6), 3–11. Google Scholar, Link Kennedy, M. M. (2016). How does professional development improve teaching? Review of Educational Research, 86, 945–980. Google Scholar, Link, ISI Kinder, D., Kubina, R., Marchand-Martella, N. E. (2005). Special education and Direct Instruction: An effective combination. Journal of Direct Instruction, 5, 1–36. Google Scholar Lipsey, M. W., Puzio, K., Yun, C., Hebert, M. A., Steinka-Fry, K., Cole, M. W., . . . Busick, M. D. (2012). Translating the statistical representation of the effects of education interventions into more readily interpretable forms (NSER 2013-3000). Washington, DC: National Center for Special Education Research, Institute of Education Sciences, U.S. Department of Education. Google Scholar MacIver, M. A., Kemper, E. (2002). The impact of Direct Instruction on elementary students’ reading achievement in an urban school district. Journal of Education for Students Placed at Risk, 7, 197–220. Google Scholar, Crossref Meyer, L. A. (1984). Long-term academic effects of the Direct Instruction Project Follow Through. The Elementary School Journal, 84, 380–394. Google Scholar, Crossref Morgan, P. L., Farkas, G., Maczuga, S. (2015). Which instructional practices most help first-grade students with and without mathematics difficulties? Educational Evaluation and Policy Analysis, 37, 184–205. Google Scholar, Link, ISI National Institute for Direct Instruction. (2016). Writings on Direct Instruction: A bibliography. Eugene, OR: Author. Retrieved from https://www.nifdi.org/docman/research/bibliography/205-di-bibliography-reference-list/file Google Scholar National Reading Panel. (2000). Report of the National Reading Panel: Teaching children to read: an evidence-based assessment of the scientific research literature on reading and its implications. Washington, DC: U.S. Department of Education. Google Scholar O’Brien, D. M., Ware, A. M. (2002). Implementing research-based reading programs in the Fort Worth Independent school district. Journal of Education for Students Placed at Risk, 7, 167–195. Google Scholar, Crossref Polanin, J. R., Tanner-Smith, E. E, Hennessy, E. A. (2016). Estimating the difference between published and unpublished effect sizes: A meta-review. Review of Educational Research, 86, 207–236. Google Scholar Popper, K. R. (1962). Conjectures and refutations: The growth of scientific knowledge. New York, NY: Basic Books. Google Scholar Przychodzin, A. M., Marchand-Martella, N. E., Martella, R. C., Azim, D. (2004). Direct Instruction mathematics programs: An overview and research summary. Journal of Direct Instruction, 4, 53–84. Google Scholar Przychodzin-Havis, A. M., Marchand-Martella, N. E., Martella, R. C., Miller, D. A., Warner, L., Leonard, B., Chapman, S. (2005). An analysis of Corrective Reading research. Journal of Direct Instruction, 5, 37–65. Google Scholar Raudenbush, S. W. (2009). Analyzing effect sizes: Random-effects models. In Cooper, H., Hedges, L. V., Valentine, J. C. (Eds.), The handbook of research synthesis and meta-analysis (pp. 295–315). New York, NY: Russell Sage Foundation. Google Scholar Ross, S. M., Nunnery, J. A., Goldfeder, E., McDonald, A., Racho, R., Hornbeck, M., Fleishman, S. (2004). Using school reform models to improve reading achievement: A longitudinal study of Direct Instruction and Success for All in an urban district. Journal of Education for Students Placed at Risk, 9, 357–388. Google Scholar, Crossref Schieffer, C., Marchand-Martella, N. E., Martella, R. C., Simonsen, F. L., Waldron-Soler, K. M. (2002). An analysis of the Reading Mastery program: Effective components and research review. Journal of Direct Instruction, 2, 87–119. Google Scholar Shadish, W. R., Cook, T. D., Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin. Google Scholar Simonsen, F., Gunter, L. (2001). Best practices in spelling instruction: A research summary. Journal of Direct Instruction, 1, 97–105. Google Scholar StataCorp. (2011). Stata Statistical Software: Release 12. College Station, TX: Author. Google Scholar Stockard, J. (2010). Promoting reading achievement and countering the “fourth-grade slump”: The impact of Direct Instruction on reading achievement in fifth grade. Journal of Education for Students Placed at Risk, 15, 218–240. Google Scholar, Crossref Stockard, J. (2011). Direct Instruction and first grade reading achievement: The role of technical support and time of implementation. Journal of Direct Instruction, 11, 31–50. Google Scholar Stockard, J. (2013). Merging the accountability and scientific research requirements of the No Child Left Behind Act: Using cohort control groups. Quality & Quantity, 47, 2225–2257. Google Scholar, Crossref Stockard, J., Wood, T. W. (2017). The threshold and inclusive approaches to determining “best available evidence.” American Journal of Evaluation, 38, 471–492. Google Scholar, Link Tallmadge, G. K. (1977). The Joint Dissemination Review Panel idea book. Washington, DC: National Institute of Education. Google Scholar Vitale, M. R., Joseph, B. L. (2008). Broadening the institutional value of Direct Instruction implemented in a low-SES elementary school: Implications for scale-up and reform. Journal of Direct Instruction, 8, 1–18. Google Scholar Vitale, M. R., Kaniuka, T. S. (2012). Adapting a multiple-baseline design rationale for evaluating instructional interventions: Implications for the adoption of Direct Instruction reading curricula for evidence-based reform. Journal of Direct Instruction, 12, 25–36. Google Scholar Watkins, C. (1996). Follow Through: Why didn’t we? Effective School Practices, 15, 56–66. Google Scholar White, W. A. T. (1988). A meta-analysis of the effects of Direct Instruction in special education. Education & Treatment of Children, 11, 364–374. Google Scholar Wood, T. W. (2014). Engelmann’s Direct Instruction: Selected writings from the past half century. Eugene, OR: NIFDI Press. Google Scholar Zhang, L. (2016). Is inquiry-based science teaching worth the effort? Science & Education, 25, 897–915. Google Scholar, Crossref

For instance, it is dangerous to predict long-term benefits from short-term results. Random assignment research has shown short-term gains from teaching “metacognitive” reading strategies (such as looking for the main idea). At the same time, cognitive theory predicts that the rate of student improvement with such interventions will not only reach a ceiling but will ultimately slow down a student’s progress in reading — an important illustration that theory (based on extensive data) is more important and useful than ad hoc data.

(o.a.: Crombag: Het oefenen van varadigheden: het juridisch practicum – T. M. Chang: Het oefenen van vaardigheden: een methode voor het analyseren en kritiseren van historische artikelen. – Van Hout en Mettes: Onderwijswijs in het leren oplossen van technische problemen. – A. D. Wolff-Albers: Beoordelen in het onderwijs W. H. F. W. Wijnen: Het diagnostisch gebruik van studietoetsen. – Wijnen: Problemen bij een selectieve propedeuse – Crombag, Roskam en Meuwese: Het meten van studiebelasting – Everwijn en Muggen: Methoden voor het meten van studietijd. – Dronkers: Functies van een bestuursstructuur voor het onderwijs – Sj. Fortuin: Het Baukastensysteem – H. M. van Strien: Doorstroming, selectie en adaptief onderwijs. – Wijnen: Blokonderwijs.)

This article describes a triarchic analysis of an aptitude–treatment interaction in a college-level introductory psychology course given to selected high school students. Of the 326 total participants, 199 were selected to be high in analytical, creative, or practical abilities, or in all 3 abilities, or in none of the 3 abilities. The selected students were placed in a course that either well matched or did not match their pattern of analytical, creative, and practical abilities. All students were assessed for memory, analytical, creative, and practical achievement. The data showed an aptitude–treatment interaction between students' varied ability patterns and the match or mismatch of these abilities to the different instructional groups. (PsycINFO Database Record (c) 2016 APA, all rights reserved)

FOR A FIELD coming somewhat late to the replication crisis party, it’s ironic that economics identified its own credibility issues early. In 1983 Edward Leamer, an economist at UCLA, published a lecture he called “Let’s Take the Con Out of Econometrics.” Leamer took his colleagues to task for the then-new practice of collecting data through observation and then fitting it to a model. In practice, Leamer said, econometricians fit their data against thousands of statistical models, found the one that worked the best, and then pretended that they were using that model all along. It’s a recipe for letting bias creep in.

maart 2022 \ contact ben at at at benwilbrink.nl

Literatuur: onderwijsonderzoek

Ben Wilbrink

Onderwijs onderzocht