Revision Manageability

Evaluator last updated June 24, 2026.

Overview

The Revision Manageability evaluator assesses whether a piece of feedback is focused enough that a student can realistically process and use it. Manageable feedback is concise, clear, and prioritizes no more than 2 issues, so that a student reading it knows immediately what to do first. The evaluator considers:

Generally no more than 4 sentences
Generally no more than 2 issues raised, with a clear priority order
Whether it includes small fixes that would have led to a complete success
Whether the student would know immediately what to do first and could act in one revision attempt

At a glance


Inputs	Teacher feedback on a student writing response
Supported grades	8–9
Rubric	Productive Coaching rubric developed by Quill.org ↗ and Leanlab Education ↗, in partnership with Anastasiya A. Lipnevich ↗

The evaluator was built and validated using the model and temperature below (other configurations will produce different results and may have lower accuracy):


Model used	GPT-5.4 (`gpt-5.4-2026-03-05`)
Temperature	1
Optimization	DSPy + GEPA (Genetic-Pareto) prompt optimization against expert-annotated labels

Getting started

Follow the Quickstart to start using this evaluator:

Access method
Evaluators playground	View in Platform ↗
GitHub	Python notebook ↗ Prompts (`system.txt` ↗ and `user.txt` ↗)

Inputs

Input	Description	Required
Student text	Student’s written response	Yes
Feedback text	Teacher feedback to be evaluated	Yes

Example input

{
  "student_text": "Some people think AI-powered pets are a good alternative to real pets because they could help around the house etc.",
  "feedback_text": "You're right, the AI pets could help around the house. Can you find some other details from the article that you could add to make your claim stronger?"
}

Output

Field	Description
Quality score	Binary judgment (1 / 0). A 1 applies when feedback is focused enough that a student could realistically process and act on it in a single revision attempt — generally four sentences or fewer, addressing no more than two clearly prioritized issues — or when the response required no revision and the feedback stays appropriately brief. A 0 signals feedback that is too dense, too long, or raises too many issues at once, or feedback so thin it omits a small fix that would have completed the response.
Reasoning	High-level summary of why the feedback received this judgment
Key features	One entry per factor driving the judgment (length, number of distinct issues, clear priority, whether the student knows the next step), each marked met (1) / not met (0) with a justification
Proposed adjustment	Suggested moves to bring the feedback into a manageable range, for developers iterating on prompts or for teachers using the feedback instructionally

Example output

{
  "reasoning": "The student response gives a very minimal claim: AI-powered pets are a good alternative because they could help around the house, but it ends with \"etc.\" and does not include enough supporting evidence. The teacher feedback responds to that main gap by affirming the existing idea and asking for additional details from the article to strengthen the claim. For manageability, the feedback is short and focused on one main revision move: add more supporting details. It does not introduce multiple separate problems or a long list of changes. The priority is fairly clear because the only issue raised is strengthening the claim with evidence from the article. A student would likely know the next step is to go back to the article and add another supporting detail or details. Therefore, this feedback is manageable in amount and scope.",
  "key_features": {
    "length": {
      "met": 1,
      "justification": "The feedback is only two sentences, so it is comfortably within the usual manageable length guideline."
    },
    "number_of_distinct_issues": {
      "met": 1,
      "justification": "The feedback raises one distinct issue: the student needs to add more details from the article to support the claim."
    },
    "clear_priority": {
      "met": 1,
      "justification": "Because only one issue is raised, the priority is clear: strengthen the claim by adding textual details."
    },
    "student_knows_next_step": {
      "met": 1,
      "justification": "The student is directly asked to find other details from the article to make the claim stronger, so the immediate next step is apparent."
    }
  },
  "proposed_adjustment": "No adjustment needed; the feedback already appears manageable.",
  "manageable_score": 1
}

Interpreting results

Output	How to use it
Revision manageability score + Reasoning	Evaluate whether your generated feedback sets students up to revise. Validate that your prompts produce focused feedback; aggregate reasoning across runs to detect drift toward longer, more exhaustive feedback
Key features + Proposed adjustment	Pinpoint and correct the specific failure mode. Require the model to prioritize the one or two highest-impact issues and cap the total it raises; target the specific failure direction

Evaluator release history

Date	Changed
June 24, 2026	First release

Feedback evaluators

Explore other evaluators that assess the quality of coaching feedback.

Quickstart

Run an evaluator in the Evaluators playground, a Python notebook, or with the SDK.

Understanding evaluators

Getting started

SDK API reference

Student-Facing Text evaluators

Feedback evaluators

Standards evaluators

Datasets

Resources

Overview

At a glance

Getting started

Inputs

Output

Interpreting results

Evaluator release history

Feedback evaluators

Quickstart

​Overview

​At a glance

​Getting started

​Inputs

​Output

​Interpreting results

​Evaluator release history

​Related topics

Feedback evaluators

Quickstart

Overview

At a glance

Getting started

Inputs

Output

Interpreting results

Evaluator release history

Related topics