> ## Documentation Index
> Fetch the complete documentation index at: https://docs.learningcommons.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Improve quality by refining prompts

> Learn how to compare evaluator scores before and after a prompt change so you know the revision actually improved quality.

Make data-driven and measurable improvements to your prompts.

## What you'll do

* Get a passage generated by the current prompt and a passage generated by the revised prompt
* Score both passages using [Grade Level Appropriateness](https://docs.learningcommons.org/evaluators/text-complexity-evaluators/grade-level-appropriateness.md), [Vocabulary Complexity](https://docs.learningcommons.org/evaluators/text-complexity-evaluators/vocabulary-complexity.md), and [Sentence Structure](https://docs.learningcommons.org/evaluators/text-complexity-evaluators/sentence-structure.md)
* Determine whether the revised prompt has improved your output's quality

## What you'll need

* Node.js or Python
* [Google API key](https://aistudio.google.com/) ↗ (needed for Grade Level Appropriateness and Vocabulary Complexity)
* [OpenAI API key](https://platform.openai.com/) ↗ (needed for Vocabulary Complexity and Sentence Structure)
* Current prompt and revised prompt, both for the same grade and task

## Steps

**1. Install the SDK**

Install the package for the language your pipeline uses:

**TypeScript**

```shell TypeScript theme={null}
npm install @learning-commons/evaluators@latest
npm install @ai-sdk/google @ai-sdk/openai # Provider adaptors for the required API keys
```

**Python**

```shell Python theme={null}
pip install --upgrade learning-commons-evaluators
```

**2. Score both prompt versions**

> [!WARNING]
>
> Inputs must be de-identified. Do not submit students' Personally
> Identifiable Information (PII) or any regulated or sensitive personal
> information.

Score each prompt's output with all 3 evaluators: [Grade Level Appropriateness](https://docs.learningcommons.org/evaluators/text-complexity-evaluators/grade-level-appropriateness.md), [Vocabulary Complexity](https://docs.learningcommons.org/evaluators/text-complexity-evaluators/vocabulary-complexity.md), and [Sentence Structure](https://docs.learningcommons.org/evaluators/text-complexity-evaluators/sentence-structure.md).

```typescript TypeScript theme={null}
import {
  GradeLevelAppropriatenessEvaluator,
  SentenceStructureEvaluator,
  VocabularyComplexityEvaluator,
} from "@learning-commons/evaluators";

const promptA =
  "Bees visit flowers to collect nectar. They carry it back to the hive and turn it into honey.";
const promptB =
  "Honeybees embark on foraging expeditions, extracting nectar from blossoms and transporting it to the colony, where enzymatic activity transforms it into honey.";

const gradeLevel = new GradeLevelAppropriatenessEvaluator({
  googleApiKey: process.env.GOOGLE_API_KEY,
});
const vocabulary = new VocabularyComplexityEvaluator({
  googleApiKey: process.env.GOOGLE_API_KEY,
  openaiApiKey: process.env.OPENAI_API_KEY,
});
const sentence = new SentenceStructureEvaluator({
  openaiApiKey: process.env.OPENAI_API_KEY,
});

async function scorePassage(text: string) {
  const [grade, words, sentences] = await Promise.all([
    gradeLevel.evaluate({ text }),
    vocabulary.evaluate({ text, grade_level: "4" }),
    sentence.evaluate({ text, grade_level: "4" }),
  ]);

  return {
    gradeBand: grade.result.grade_band, // "K-1" | "2-3" | "4-5" | "6-8" | "9-10" | "11-12"
    vocabulary: words.result.complexity_score, // "slightly_complex" | "moderately_complex" | "very_complex" | "exceedingly_complex"
    sentence: sentences.result.complexity_score,
  };
}

const before = await scorePassage(promptA);
const after = await scorePassage(promptB);

console.log("Current prompt", before);
console.log("Revised prompt", after);
```

```python Python theme={null}
import os

from learning_commons_evaluators import (
  GradeLevelAppropriatenessEvaluator,
  SentenceStructureEvaluator,
  VocabularyComplexityEvaluator,
)

prompt_a = (
  "Bees visit flowers to collect nectar. They carry it back to the hive "
  "and turn it into honey."
)
prompt_b = (
  "Honeybees embark on foraging expeditions, extracting nectar from "
  "blossoms and transporting it to the colony, where enzymatic activity "
  "transforms it into honey."
)

grade_level = GradeLevelAppropriatenessEvaluator(
  google_api_key=os.environ["GOOGLE_API_KEY"],
)
vocabulary = VocabularyComplexityEvaluator(
  google_api_key=os.environ["GOOGLE_API_KEY"],
  openai_api_key=os.environ["OPENAI_API_KEY"],
)
sentence = SentenceStructureEvaluator(
  openai_api_key=os.environ["OPENAI_API_KEY"],
)

def score_passage(text):
  grade = grade_level.evaluate_sync(text=text).result
  words = vocabulary.evaluate_sync(text=text, grade_level=4).result
  sentences = sentence.evaluate_sync(text=text, grade_level=4).result

  return {
    "grade_band": grade.grade_band,  # "K-1" | "2-3" | "4-5" | "6-8" | "9-10" | "11-12"
    "vocabulary": words.complexity_score,  # "slightly_complex" | "moderately_complex" | "very_complex" | "exceedingly_complex"
    "sentence": sentences.complexity_score,
  }

before = score_passage(prompt_a)
after = score_passage(prompt_b)

print("Current prompt", before)
print("Revised prompt", after)
```

**3. Determine whether the revised prompt should be shipped**

Ship the revised prompt only if its output's:

* Evaluated grade level is at or below your target
* [Vocabulary Complexity](https://docs.learningcommons.org/evaluators/text-complexity-evaluators/vocabulary-complexity.md) and [Sentence Structure](https://docs.learningcommons.org/evaluators/text-complexity-evaluators/sentence-structure.md) scores have not gotten more complex

> [!NOTE]
>
> If the revised prompt is meant to produce a more difficult text, be sure to reverse the check and ensure that your Vocabulary Complexity and Sentence Structure scores HAVE gotten more complex.

```typescript TypeScript theme={null}
const GRADE_BANDS = ["K-1", "2-3", "4-5", "6-8", "9-10", "11-12"] as const;
const COMPLEXITY = [
  "slightly_complex",
  "moderately_complex",
  "very_complex",
  "exceedingly_complex",
] as const;
const TARGET_BAND = "4-5";

const bandOk =
  GRADE_BANDS.indexOf(after.gradeBand) <= GRADE_BANDS.indexOf(TARGET_BAND);
const vocabularyOk =
  COMPLEXITY.indexOf(after.vocabulary) <= COMPLEXITY.indexOf(before.vocabulary);
const sentenceOk =
  COMPLEXITY.indexOf(after.sentence) <= COMPLEXITY.indexOf(before.sentence);

if (bandOk && vocabularyOk && sentenceOk) {
  console.log("Scores held. The revised prompt can ship.");
} else {
  console.log("A dimension moved the wrong way. Keep the current prompt.");
  console.log({ bandOk, vocabularyOk, sentenceOk });
}
```

```python Python theme={null}
GRADE_BANDS = ["K-1", "2-3", "4-5", "6-8", "9-10", "11-12"]
COMPLEXITY = [
  "slightly_complex",
  "moderately_complex",
  "very_complex",
  "exceedingly_complex",
]
TARGET_BAND = "4-5"

band_ok = GRADE_BANDS.index(after["grade_band"]) <= GRADE_BANDS.index(TARGET_BAND)
vocabulary_ok = COMPLEXITY.index(after["vocabulary"]) <= COMPLEXITY.index(before["vocabulary"])
sentence_ok = COMPLEXITY.index(after["sentence"]) <= COMPLEXITY.index(before["sentence"])

if band_ok and vocabulary_ok and sentence_ok:
  print("Scores held. The revised prompt can ship.")
else:
  print("A dimension moved the wrong way. Keep the current prompt.")
  print({
    "band_ok": band_ok,
    "vocabulary_ok": vocabulary_ok,
    "sentence_ok": sentence_ok,
  })
```

For any dimension that fails its check, change the prompt to address that evaluator's `reasoning`.

To monitor one dimension of a sample over time, see [Monitor quality in production](https://docs.learningcommons.org/evaluators/getting-started/tutorials/monitor-quality-in-production.md).

> [!TIP]
>
> A single passage's scores can move for reasons that have nothing to do with the prompt.
>
> Run several evaluations for each passage-dimension pair before moving forward.

## Related topics

> **[Resource Library - Improve quality by refining prompts](https://learningcommons.org/resources/refine-prompts-improve-quality/)**
>
> Score outputs from before and after a prompt change on the same rubric so
> you can see what shifted.

> **[Quickstart](https://docs.learningcommons.org/evaluators/getting-started/quickstart.md)**
>
> Run an evaluator in the Evaluators playground, a Python notebook, or with
> the SDK.

> **[Tutorial: Vet content quality in production](https://docs.learningcommons.org/evaluators/getting-started/tutorials/vet-content-quality-in-production.md)**
>
> Score each output and rewrite anything above your grade-band target.

> **[Tutorial: Monitor quality in production](https://docs.learningcommons.org/evaluators/getting-started/tutorials/monitor-quality-in-production.md)**
>
> Track grade band across a sample of outputs and catch a shift over time.