Actionability

Evaluator last updated June 24, 2026.

Overview

The Actionability evaluator assesses whether a piece of feedback gives the student a clear, usable next step they can reasonably act on without additional clarification. The evaluator considers:

Presence of a directive verb (e.g., add, replace, clarify, explain, revise) or focused question that points somewhere specific
Whether feedback goes beyond evaluative comments (e.g., “good job,” “this is incomplete”)
Clarity of the target (is it clear what should be revised?)
Specificity of the next move (a concrete directive rather than a vague instruction)
Whether the student could reasonably act without further clarification or more information
Whether a next step is warranted at all (a response that didn’t require revision should not draw a forced directive)

At a glance


Input type	Teacher feedback on a student writing response
Supported grades	8–9
Rubric	Productive Coaching rubric developed by Quill.org ↗ and Leanlab Education ↗, in partnership with Anastasiya A. Lipnevich ↗

The evaluator was built and validated using the model and temperature below (other configurations will produce different results and may have lower accuracy):


Model used	GPT-5.4
Temperature	1.0
Optimization	DSPy + GEPA (Genetic-Pareto) prompt optimization against expert-annotated labels

Getting started

Follow the Quickstart to start using this evaluator:

Access method
Evaluators Playground	View in the Learning Commons Platform ↗
Python notebook	View in GitHub ↗
Prompts	View in GitHub: `system.txt` ↗ and `user.txt` ↗

Inputs

Input	Description	Required
Student text	Student’s written response	Yes
Feedback text	Teacher feedback to be evaluated	Yes

Example input

{
  "student_text": "Some people think AI-powered pets are a good alternative to real pets because they could help around the house etc.",
  "feedback_text": "You're right, the AI pets could help around the house. Can you find some other details from the article that you could add to make your claim stronger?"
}

Output

Field	Description
Actionable revision score	Binary judgment (1 / 0). A 1 applies when feedback names a concrete action the student can take — add, replace, clarify, explain, ask, revise — or appropriately omits a next step when the response required no revision. A 0 signals feedback that is purely evaluative, too vague to act on, or too thin to direct revision.
Reasoning	High-level summary of why the feedback received this judgment
Key features	One entry per factor driving the judgment (directive verb or focused question, clarity of the target, specificity of the next move, whether the student could reasonably act without further clarification), each marked met (1) / not met (0) with a justification
Proposed adjustment	Suggested moves to strengthen the next step, for developers iterating on prompts or for teachers using the directive instructionally

Example output

{
  "reasoning": "The student gives a vague reason ('less of a responsibility') without specific text-based support, so revision is needed. The feedback affirms the idea and adds related context, but names no action the student could take — it does not provide a usable next step.",
  "key_features": {
    "directive_verb_or_focused_question": {
      "met": "no",
      "justification": "Affirms the student ('That's right') and adds a comment, but contains no directive verb or focused question prompting revision."
    },
    "clarity_of_target": {
      "met": "no",
      "justification": "Does not identify any part of the response to revise."
    },
    "specificity_of_next_move": {
      "met": "no",
      "justification": "Names no concrete action the student could take."
    },
    "reasonable_student_action": {
      "met": "no",
      "justification": "The student is left without a clear sense of what to change or add."
    }
  },
  "proposed_adjustment": "Add a directive verb or focused question. Example: 'That's a reason from the article. Can you add a specific example of what the article says makes AI pets less of a responsibility?'",
  "actionable_revision_score": "no"
}

Interpreting results

Output	How to use it
Actionable revision score + Reasoning	Evaluate whether your generated feedback is actually driving revision. Validate that your prompts produce feedback students can act on; aggregate reasoning across runs to detect drift toward describe-only, evaluative feedback
Key features + Proposed adjustment	Pinpoint and correct the specific failure. Require the model to include a directive verb or focused question before closing; surface 0-rated outputs to flag where students may need teacher follow-up

Evaluator release history

Date	Changed
June 24, 2026	First release

Understanding evaluators

Getting started

SDK API Reference

Literacy evaluators

Feedback evaluators

Standards evaluators

Datasets

Resources

Overview

At a glance

Getting started

Inputs

Output

Interpreting results

Evaluator release history

​Overview

​At a glance

​Getting started

​Inputs

​Output

​Interpreting results

​Evaluator release history

Overview

At a glance

Getting started

Inputs

Output

Interpreting results

Evaluator release history