> ## Documentation Index
> Fetch the complete documentation index at: https://docs.learningcommons.org/llms.txt
> Use this file to discover all available pages before exploring further.
# Actionability
> Reference documentation for the Actionability feedback evaluator.
[Evaluator last updated June 24, 2026.](#evaluator-release-history)
## Overview
The Actionability evaluator assesses whether a piece of feedback gives the student a clear, usable next step they can reasonably act on without additional clarification.
The evaluator considers:
* Presence of a directive verb (e.g., add, replace, clarify, explain, revise) or focused question that points somewhere specific
* Whether feedback goes beyond evaluative comments (e.g., "good job," "this is incomplete")
* Clarity of the target (is it clear *what* should be revised?)
* Specificity of the next move (a concrete directive rather than a vague instruction)
* Whether the student could reasonably act without further clarification or more information
* Whether a next step is warranted at all (a response that didn't require revision should not draw a forced directive)
## At a glance
| | |
| :------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Input type** | Teacher feedback on a student writing response |
| **Supported grades** | 8–9 |
| **Rubric** | Productive Coaching rubric developed by [Quill.org](https://www.quill.org/) ↗ and [Leanlab Education](https://www.leanlabeducation.org/) ↗, in partnership with [Anastasiya A. Lipnevich](https://www.anastasiyalipnevich.com/) ↗ |
The evaluator was built and validated using the model and temperature below (other configurations will produce different results and may have lower accuracy):
| | |
| :--------------- | :------------------------------------------------------------------------------- |
| **Model used** | GPT-5.4 |
| **Temperature** | 1.0 |
| **Optimization** | DSPy + GEPA (Genetic-Pareto) prompt optimization against expert-annotated labels |
## Getting started
Follow the [Quickstart](/evaluators/getting-started/quickstart) to start using this evaluator:
| Access method | |
| :------------------------ | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Evaluators Playground** | [View in the Learning Commons Platform](https://platform.learningcommons.org/apps/evaluators/playground?utm_source=docs\&utm_medium=evaluators) ↗ |
| **Python notebook** | [View in GitHub](https://github.com/learning-commons-org/evaluators/tree/main/evals/feedback/productive-coaching-writing-feedback/actionable-revision/example_notebook.ipynb?utm_source=docs\&utm_medium=evaluators) ↗ |
| **Prompts** | View in GitHub: [`system.txt`](https://github.com/learning-commons-org/evaluators/blob/main/evals/feedback/productive-coaching-writing-feedback/actionable-revision/system.txt?utm_source=docs\&utm_medium=evaluators) ↗ and [`user.txt`](https://github.com/learning-commons-org/evaluators/blob/main/evals/feedback/productive-coaching-writing-feedback/actionable-revision/user.txt?utm_source=docs\&utm_medium=evaluators) ↗ |
## Inputs
Inputs must be de-identified. Do not submit student PII or any regulated or
sensitive personal information.
| Input | Description | Required |
| :---------------- | :----------------------------------------------- | :------- |
| **Student text** | Student's constructed-response writing | Yes |
| **Feedback text** | Teacher or AI-generated feedback to be evaluated | Yes |
```json Example input theme={null}
{
"student_text": "Some people think AI-powered pets are a good alternative to real pets because they could help around the house etc.",
"feedback_text": "You're right, the AI pets could help around the house. Can you find some other details from the article that you could add to make your claim stronger?"
}
```
## Output
| Field | Description |
| :---------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Actionable revision score** | Binary judgment (1 / 0). A 1 applies when feedback names a concrete action the student can take — add, replace, clarify, explain, ask, revise — or appropriately omits a next step when the response required no revision. A 0 signals feedback that is purely evaluative, too vague to act on, or too thin to direct revision. |
| **Reasoning** | High-level summary of why the feedback received this judgment |
| **Key features** | One entry per factor driving the judgment (directive verb or focused question, clarity of the target, specificity of the next move, whether the student could reasonably act without further clarification), each marked met (1) / not met (0) with a justification |
| **Proposed adjustment** | Suggested moves to strengthen the next step, for developers iterating on prompts or for teachers using the directive instructionally |
```json Example output theme={null}
{
"reasoning": "The student gives a vague reason ('less of a responsibility') without specific text-based support, so revision is needed. The feedback affirms the idea and adds related context, but names no action the student could take — it does not provide a usable next step.",
"key_features": {
"directive_verb_or_focused_question": {
"met": "no",
"justification": "Affirms the student ('That's right') and adds a comment, but contains no directive verb or focused question prompting revision."
},
"clarity_of_target": {
"met": "no",
"justification": "Does not identify any part of the response to revise."
},
"specificity_of_next_move": {
"met": "no",
"justification": "Names no concrete action the student could take."
},
"reasonable_student_action": {
"met": "no",
"justification": "The student is left without a clear sense of what to change or add."
}
},
"proposed_adjustment": "Add a directive verb or focused question. Example: 'That's a reason from the article. Can you add a specific example of what the article says makes AI pets less of a responsibility?'",
"actionable_revision_score": "no"
}
```
## Interpreting results
| Output | How to use it |
| :---------------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Actionable revision score + Reasoning** | Evaluate whether your generated feedback is actually driving revision. Validate that your prompts produce feedback students can act on; aggregate reasoning across runs to detect drift toward describe-only, evaluative feedback |
| **Key features + Proposed adjustment** | Pinpoint and correct the specific failure. Require the model to include a directive verb or focused question before closing; surface 0-rated outputs to flag where students may need teacher follow-up |
## Accuracy and validation
This evaluator is provided as Early access. Reported metrics come from a small
held-out test split (19 examples) with wide confidence intervals and should be
read as directional. Validation testing is ongoing.
We assessed performance against [Quill.org](https://www.quill.org/) ↗ classroom writing data (45 labeled pairs; 13 train / 13 validation / 19 test) — expert-annotated student-response and teacher-feedback pairs labeled by [Leanlab Education](https://www.leanlabeducation.org/) ↗ using the Productive Coaching rubric.
| Metric | Description | Result |
| :------------------------- | :-------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------ |
| **Accuracy** | How accurately the evaluator's actionable-revision score matches expert annotations on the held-out test split. | 95% (naive baseline: 95%) |
| **Macro-F1** | Macro-F1 averaged over both classes on the held-out test split. | 94% (naive baseline: 94%) |
| **Model selection signal** | Mean optimized performance of GPT-5.4 across the 5 feedback dimensions, which drove model selection. | GPT-5.4 had the best mean optimized macro-F1 (≈0.81) and accuracy (≈0.82) across the five feedback dimensions |
On this dimension, GEPA optimization did not improve GPT-5.4 over its naive
baseline on the held-out test set — both reached 95% accuracy. GPT-5.4 was
selected for its strong mean performance across the full suite rather than its
margin on this single dimension.
## Evaluator release history
| Date | Changed |
| ------------- | ------------- |
| June 24, 2026 | First release |