Strength Acknowledgement

Evaluator last updated June 24, 2026.

Overview

The Strength Acknowledgement evaluator assesses whether a piece of feedback names something specific and authentic that the student did well. The evaluator considers:

Whether praise is authentic or reflexive
Whether feedback is specific or generic
Whether feedback is anchored to evidence – i.e., a feature of the student’s response
Whether it uses process-vs.-trait framing – i.e., praising what the student did rather than a fixed ability
Whether acknowledgment is warranted – e.g., “IDK” should not draw false praise

At a glance


Input type	Teacher feedback on a student writing response
Supported grades	8–9
Rubric	Productive Coaching rubric developed by Quill.org ↗ and Leanlab Education ↗, in partnership with Anastasiya A. Lipnevich ↗

The evaluator was built and validated using the model and temperature below (other configurations will produce different results and may have lower accuracy):


Model used	GPT-5.4
Temperature	1.0
Optimization	DSPy + GEPA (Genetic-Pareto) prompt optimization against expert-annotated labels

Getting started

Follow the Quickstart to start using this evaluator:

Access method
Evaluators Playground	View in the Learning Commons Platform ↗
Python notebook	View in GitHub ↗
Prompts	View in GitHub: `system.txt` ↗ and `user.txt` ↗

Inputs

Inputs must be de-identified. Do not submit student PII or any regulated or sensitive personal information.

Input	Description	Required
Student text	Student’s constructed-response writing	Yes
Feedback text	Teacher or AI-generated feedback to be evaluated	Yes

Output

Field	Description
Acknowledges strength score	Binary judgment (1 / 0). A 1 applies when feedback names a specific, accurate strength tied to what the student wrote — or appropriately refrains from praise when the response warrants none. A 0 signals generic, unanchored, or inaccurate praise.
Reasoning	High-level summary of why the feedback received this judgment
Key features	One entry per factor driving the judgment (presence of praise, specificity, anchoring to evidence, process-vs.-trait framing), each marked met (1) / not met (0) with a justification
Proposed adjustment	Suggested moves to strengthen the feedback’s acknowledgment, for developers iterating on prompts

Example output

{
  "reasoning": "The student response includes a concrete, relevant idea: AI-powered pets could help around the house. Although the phrase etc. is vague, the helping-around-the-house idea is a promising reason that could be developed with evidence. The teacher feedback explicitly acknowledges that specific idea by saying, You're right, the AI pets could help around the house. This is a direct, response-specific acknowledgment of a strength or valid point in the student's writing. The follow-up question then invites the student to strengthen the claim by adding more details from the article, which builds on that identified strength rather than ignoring it. Because the acknowledgment is specific, grounded in the student's actual wording, and framed as a next step for developing the idea, the feedback meets the criterion.",
  "key_features": {
    "presence_of_praise": {
      "met": 1,
      "justification": "The feedback explicitly affirms a specific student idea with \"You're right, the AI pets could help around the house.\" This is authentic acknowledgment rather than generic praise."
    },
    "specificity": {
      "met": 1,
      "justification": "The acknowledgment names the particular idea from the student response: that AI pets could help around the house. It is not vague or interchangeable with feedback on any essay."
    },
    "anchoring_to_evidence": {
      "met": 1,
      "justification": "The feedback is clearly tied to wording that appears in the student text, specifically the claim about helping around the house. It is grounded in the student's actual response."
    },
    "process_vs_trait_framing": {
      "met": 1,
      "justification": "The feedback focuses on the student's claim and a revision move—adding details from the article to strengthen it—rather than praising a fixed trait. This frames the strength as something to build on through writing process."
    }
  },
  "proposed_adjustment": "No adjustment needed; the feedback already meets the criterion. If desired, the teacher could make it even stronger by naming one especially useful detail from the article as a model for expansion.",
  "acknowledges_strength_score": 1
}

Interpreting results

Output	How to use it
Acknowledges strength score + Reasoning	Validate that your AI-generated feedback is building student awareness, not just adding polite filler. Ensure that it includes specific, earned praise. Aggregate reasoning across runs to detect drift toward generic and/or polite openers.
Key features + Proposed adjustment	Pinpoint and correct the specific failure mode. Adjust prompts to require the model to name a specific feature of the student’s work before any directive; target the missing feature directly

Accuracy and validation

This evaluator is provided as Early access. Reported metrics come from small held-out test splits with wide confidence intervals and should be read as directional. Validation testing is ongoing.

We assessed performance against Quill.org ↗ classroom writing data (82 labeled pairs; 24 train / 24 validation / 34 test) — expert-annotated student-response and teacher-feedback pairs labeled by Leanlab Education ↗ using the Productive Coaching rubric.

Metric	Description	Result
Accuracy	How accurately the evaluator’s acknowledges-strength score matches expert annotations on the held-out test split.	71% (naive baseline: 74%)
Macro-F1	Macro-F1 averaged over both classes on the held-out test split.	69% (naive baseline: 73%)
Model selection signal	Mean optimized performance of GPT-5.4 across the 5 feedback dimensions, which drove model selection.	GPT-5.4 had the best mean optimized macro-F1 (≈0.81) and accuracy (≈0.82) across the five feedback dimensions

On this dimension, GEPA optimization did not improve GPT-5.4 over its naive baseline on the held-out test set. GPT-5.4 was selected for its strong mean performance across the full suite rather than its margin on this single dimension.

Evaluator release history

Date	Changed
June 24, 2026	First release

Understanding evaluators

Getting started

SDK API Reference

Literacy evaluators

Feedback evaluators

Standards evaluators

Datasets

Resources

Overview

At a glance

Getting started

Inputs

Output

Interpreting results

Accuracy and validation

Evaluator release history

​Overview

​At a glance

​Getting started

​Inputs

​Output

​Interpreting results

​Accuracy and validation

​Evaluator release history

Overview

At a glance

Getting started

Inputs

Output

Interpreting results

Accuracy and validation

Evaluator release history