> ## Documentation Index
> Fetch the complete documentation index at: https://docs.learningcommons.org/llms.txt
> Use this file to discover all available pages before exploring further.
# Withholding Answers
> Reference documentation for the Withholding Answers feedback evaluator.
export const PiiWarning = () => {
return
Inputs must be de-identified. Do not submit students' Personally
Identifiable Information (PII) or any regulated or sensitive personal
information.
;
};
[Evaluator last updated July 29, 2026.](#evaluator-release-history)
## Overview
The Withholding Answers evaluator assesses whether a piece of teacher feedback guides the student toward a revision without doing the cognitive work for them:
* Scaffolds through hints, guiding questions, or naming the kind of content or evidence that is missing
* Doesn't just supply the evidence, rewrite the response, or otherwise provide a complete or near-complete answer
The evaluator considers whether the feedback:
* Points toward where to find evidence vs. supplying the actual evidence
* Prompts the student to revise rather than rewriting or modeling the full response
* Leaves core thinking (selecting, explaining, or wording the evidence) to the student
## At a glance
| | |
| :------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Input type** | Teacher feedback on a student writing response |
| **Supported grades** | 8–9 |
| **Rubric** | Productive Coaching rubric developed by [Quill.org](https://www.quill.org/) ↗ and [Leanlab Education](https://www.leanlabeducation.org/) ↗, in partnership with [Anastasiya A. Lipnevich](https://www.anastasiyalipnevich.com/) ↗ |
The evaluator was built and validated using the model and temperature below (other configurations will produce different results and may have lower accuracy):
| | |
| :--------------- | :------------------------------------------------------------------------------- |
| **Model used** | GPT-5.4 |
| **Temperature** | 1.0 |
| **Optimization** | DSPy + GEPA (Genetic-Pareto) prompt optimization against expert-annotated labels |
## Getting started
Follow the [Quickstart](/evaluators/getting-started/quickstart) to start using this evaluator:
| Access method | |
| :------------------------ | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Evaluators Playground** | [View in the Learning Commons Platform](https://platform.learningcommons.org/apps/evaluators/playground?utm_source=docs\&utm_medium=evaluators) ↗ |
| **Python notebook** | [View in GitHub](https://github.com/learning-commons-org/evaluators/tree/main/evals/feedback/productive-coaching-writing-feedback/withholding-answers/example_notebook.ipynb?utm_source=docs\&utm_medium=evaluators) ↗ |
| **Prompts** | View in GitHub: [`system.txt`](https://github.com/learning-commons-org/evaluators/blob/main/evals/feedback/productive-coaching-writing-feedback/withholding-answers/system.txt?utm_source=docs\&utm_medium=evaluators) ↗ and [`user.txt`](https://github.com/learning-commons-org/evaluators/blob/main/evals/feedback/productive-coaching-writing-feedback/withholding-answers/user.txt?utm_source=docs\&utm_medium=evaluators) ↗ |
## Inputs
| Input | Description | Required |
| :---------------- | :------------------------------------------------ | :------- |
| **Student text** | Student's written response | Yes |
| **Feedback text** | Teacher- or AI-generated feedback to be evaluated | Yes |
## Output
| Field | Description |
| :---------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Withholding answers score** | Binary judgment (1 / 0). A 1 applies when the feedback guides revision through hints, guiding questions, or by naming what kind of content or evidence is missing — without supplying it — or when the student's response didn't require revision and there was no answer to give away. A 0 signals feedback that does the work for the student: supplying the specific evidence, rewriting the response, or otherwise providing a complete or near-complete answer. |
| **Reasoning** | High-level summary of why the feedback received this judgment |
| **Key features** | One entry per factor driving the judgment (points toward evidence without supplying, prompts revision without rewriting, leaves core thinking to student), each marked met (1) / not met (0) with a justification |
| **Proposed adjustment** | Suggested moves to convert answer-giving into scaffolding, for developers iterating on prompts |
```json Example output theme={null} theme={null}
{
"reasoning": "The feedback asks the student to explain and provide examples of why an AI pet costs less, but does not supply those examples or specific costs itself. The cognitive work — locating and articulating the evidence — is left to the student.",
"key_features": {
"points_toward_evidence_without_supplying": {
"met": 1,
"justification": "Asks the student to explain and give examples of why cost is lower, without naming the specific costs or quoting the text."
},
"prompts_revision_without_rewriting": {
"met": 1,
"justification": "Prompts the student to revise; it does not rewrite or model the response for them."
},
"leaves_core_thinking_to_student": {
"met": 1,
"justification": "Selecting, explaining, and wording the evidence is left to the student."
}
},
"proposed_adjustment": "None needed for withholding-answers. The feedback scaffolds without completing the task.",
"withholding_answers_score": 1
}
```
## Interpreting results
| Output | How to use it |
| :---------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Withholding answers score + Reasoning** | Validate that your AI-generated feedback is coaching students through the work rather than doing it for them. Aggregate reasoning across runs to detect answer-giving drift — models tuned to be maximally helpful tend to drift toward supplying evidence or modeling full revisions as they're tuned for helpfulness. |
| **Key features + Proposed adjustment** | Pinpoint exactly where a comment crosses from guiding into answer-giving. Adjust prompts to require the model to point toward what's missing (the kind of evidence, or where to look) rather than supply it, and to withhold any full or near-full revision. |
## Evaluator release history
| Date | Changed |
| ------------- | ------------- |
| July 29, 2026 | First release |