Student Response Specificity

Evaluator last updated June 24, 2026.

Overview

The Student Response Specificity evaluator assesses whether a piece of feedback is clearly based on the student’s specific response — i.e., referencing, building on, or responding to the student’s own idea, wording, or use of evidence. In short, it assesses whether the feedback demonstrates an accurate understanding of what the student wrote. The evaluator considers whether:

There is a specific reference to the student’s work (the feedback names or builds on the student’s particular idea, wording, or evidence)
Feedback is generic or template-based (a stock phrase that could apply to any response)
Feedback is based on an accurate understanding and interpretation of the student’s response
Feedback addresses something the student did not actually say or attempt

At a glance


Inputs	Teacher feedback on a student writing response
Supported grades	8–9
Rubric	Productive Coaching rubric developed by Quill.org ↗ and Leanlab Education ↗, in partnership with Anastasiya A. Lipnevich ↗

The evaluator was built and validated using the model and temperature below (other configurations will produce different results and may have lower accuracy):


Model used	GPT-5.4 (`gpt-5.4-2026-03-05`)
Temperature	1
Optimization	DSPy + GEPA (Genetic-Pareto) prompt optimization against expert-annotated labels

Getting started

Follow the Quickstart to start using this evaluator:

Access method
Evaluators playground	View in Platform ↗
GitHub	Python notebook ↗ Prompts (`system.txt` ↗ and `user.txt` ↗)

Inputs

Input	Description	Required
Student text	Student’s written response	Yes
Feedback text	Teacher feedback to be evaluated	Yes

Example input

{
  "student_text": "Some people think AI-powered pets are a good alternative to real pets because they could help around the house etc.",
  "feedback_text": "You're right, the AI pets could help around the house. Can you find some other details from the article that you could add to make your claim stronger?"
}

Output

Field	Description
Quality score	Binary judgment (1 / 0). A 1 applies when feedback references, builds on, or responds to the student’s own idea, wording, or use of evidence and demonstrates accurate understanding of what they wrote. A 0 signals feedback that is generic or template-like, built on a misinterpretation, or addressed to something the student did not say or attempt.
Reasoning	High-level summary of why the feedback received this judgment
Key features	One entry per factor driving the judgment (specific reference to the student’s work, not generic or template-based, engagement based on accurate understanding), each marked met (1) / not met (0) with a justification
Proposed adjustment	Suggested moves to anchor the feedback more clearly in the student’s response, for developers iterating on prompts or for teachers using the feedback instructionally

Example output

{
  "reasoning": "The student's specific claim is that AI-powered pets are a good alternative because \"they could help around the house.\" The teacher feedback directly echoes that exact idea by saying \"the AI pets could help around the house,\" which shows a clear connection to the student's actual wording and reasoning. The follow-up suggestion to add other details from the article is tied to that existing claim rather than being completely generic. The feedback also accurately understands the student's point and does not misread or shift it to a different argument. Therefore, the feedback is sufficiently anchored in the student response.",
  "key_features": {
    "specific_reference_to_student_work": {
      "met": 1,
      "justification": "The feedback explicitly repeats the student's reason that AI pets \"could help around the house,\" directly referencing the student's actual claim."
    },
    "not_generic_or_template_based": {
      "met": 1,
      "justification": "Although it includes a common suggestion to add details, it is not purely generic because it first identifies the student's specific idea about helping around the house."
    },
    "engage_based_on_accurate_understanding": {
      "met": 1,
      "justification": "The feedback accurately understands and responds to the student's stated reason rather than introducing a different idea."
    }
  },
  "proposed_adjustment": "No adjustment needed; the feedback already meets the criterion. It could be even stronger by asking for a specific example of how the pets help around the house.",
  "anchored_in_student_response_score": 1
}

Interpreting results

Output	How to use it
Anchored in student response score + Reasoning	Evaluate whether your generated feedback is responding to this student’s actual work or to a generic version of the task. Validate that your prompts produce responsive feedback; aggregate reasoning across runs to detect drift toward safe, generic phrasing
Key features + Proposed adjustment	Pinpoint and correct the specific failure mode. Require the model to quote or restate a specific element of the student’s response before offering guidance; target the specific failure mode

Evaluator release history

Date	Changed
June 24, 2026	First release

Feedback evaluators

Explore other evaluators that assess the quality of coaching feedback.

Quickstart

Run an evaluator in the Evaluators playground, a Python notebook, or with the SDK.

Understanding evaluators

Getting started

SDK API reference

Student-Facing Text evaluators

Feedback evaluators

Standards evaluators

Datasets

Resources

Overview

At a glance

Getting started

Inputs

Output

Interpreting results

Evaluator release history

Feedback evaluators

Quickstart

​Overview

​At a glance

​Getting started

​Inputs

​Output

​Interpreting results

​Evaluator release history

​Related topics

Feedback evaluators

Quickstart

Overview

At a glance

Getting started

Inputs

Output

Interpreting results

Evaluator release history

Related topics