Overview
The Math Alignment evaluator checks a math question against a standard’s individual learning components — not just against the standard’s label. It reports which of a standard’s components the question actually measures.The Math Alignment evaluator judges whether a question is the right math for
a standard. The Math Visual Correctness evaluator (coming soon) judges whether
a math visual is mathematically correct.
At a glance
The evaluator was built and validated using the model and temperature below (other configurations will produce different results and may have lower accuracy):
Getting started
Follow the Quickstart to start using this evaluator:Inputs
The evaluator supports 3 modes:
Output
The evaluator reduces alignment to a binary judgment (plus rationale) per learning component, and is not validated for grading, assessment, or placement decisions.Treat outputs as directional signals, and keep a human in the loop – especially for borderline cases.
Interpreting results
The evaluator assesses standards alignment based on how many of a standard’s learning components a question meets (i.e., Aligned count / Total count). Users should interpret the counts while keeping in mind the learning components a question was meant to target. An Aligned count of 2 out of a Total count of 5 is ambiguous information on its own. If those 2 Aligned count learning components include the ones a question was meant to target, you can conclude the question is aligned. Otherwise, you can conclude the question is not aligned.Example: A question that asks students to find the area of a rectangle by
multiplying its sides’ lengths is commonly tagged to Common Core 3.MD.C.7.
However, that question meets only one of the 4 learning components that make up
that standard.At the parent-code level the question looks aligned; at the learning-component
level, it covers a quarter of the standard.