Beyond the final answer

The artifact and the process

A correct answer can be useful. It does not always reveal how the answer was produced, what the learner can explain, or what they would do in a nearby case.

Generative systems make this distinction hard to ignore. A fluent artifact can be produced through several routes, only some of which require the learner to construct the relationships expressed in it. This essay proposes an assessment question; it does not report a measured change in student understanding.

What we are trying to see

If the educational aim includes reasoning, then an evaluation needs evidence of reasoning. That might include predictions, explanations, revisions, or transfer to a new case. None is automatically a transparent measure. Each needs a task and an interpretation rule.

The note on relational-looking output makes the narrower point: the structure of an explanation should not silently become a claim about the independent capacity of its author.

Keep the learner in the loop

A research direction is to design interactions in which the learner has something substantive left to generate. That is different from making a task artificially tedious. The remaining work should serve the learning aim.

The ICAP framework offers a related vocabulary for thinking about overt engagement activities; it is background reading for this inquiry, not validation of the proposed tutor.

An assessment question worth keeping

What would a learner need to show for us to believe that they can use the idea beyond the artifact in front of us?

The proposed observe-first tutor is one place to explore that question. The benchmark addresses a preceding question: whether the tutor responds to differences in demonstrated reasoning at all.

References & reading

1. Chi & Wylie (2014). The ICAP Framework: Linking Cognitive Engagement to Active Learning Outcomes. Educational Psychologist. ↗