NotebookGrowing

Structure is not the same as factual correctness

Two dimensions that a reasoning-adaptation benchmark needs to keep separate.

Separate the axes

An explanation may connect several elements coherently while relying on a false premise. Another may state one correct fact without connecting it to anything else. Calling the first “more structured” does not make it more factually correct.

This distinction matters when constructing SOLO-informed contrasts. If structure and correctness always rise together, the tutor’s response cannot tell us which property it is responding to.

A construction rule to investigate

For each task, write down the factual claims and relationships in a candidate explanation separately. Audit which claims are correct, which elements are relevant, and which relationships the learner explicitly demonstrates.

DimensionQuestion for the item audit
Task factsWhat information is available to the tutor?
CorrectnessWhich assertions are true of the program?
StructureWhich relevant elements are identified or related?
ExpressionDoes wording, length, or confidence introduce a cue?

This table is a design aid, not a validated rating rubric.

A limit worth preserving

The benchmark should report the controlled contrasts it actually constructs. It should not claim a general separation of structure and correctness merely because the item labels are different.

References & reading

  1. Biggs & Collis (1982). Evaluating the Quality of Learning: The SOLO Taxonomy. Academic Press. ↗
Continue the thread

Connected thinking.

Return to the notebook →

Follow a question.

Search notes, projects, essays, and research streams.

Open full search →