Overview
The Revision Accuracy evaluator assesses whether a piece of feedback correctly identifies whether a student’s response has met the task goal and thus does not require further revision. The evaluator considers:- Accurate task assessment (whether the response meets the task’s relevant requirements)
- Revision-need identification (distinguishing between feedback that should direct revision and feedback that should signal “move on”)
- Appropriate signal when the task is complete (signaling the student can move on when the goal is met)
At a glance
The evaluator was built and validated using the model and temperature below (other configurations will produce different results and may have lower accuracy):
Getting started
Follow the Quickstart to start using this evaluator:Inputs
Example input
Output
Example output
Interpreting results
Evaluator release history
Related topics
Feedback evaluators
Explore other evaluators that assess the quality of coaching feedback.
Quickstart
Run an evaluator in the Evaluators playground, a Python notebook, or with
the SDK.