> ## Documentation Index
> Fetch the complete documentation index at: https://docs.learningcommons.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Vet content quality in production

> Learn how to score every generated output in your pipeline and rewrite anything that misses your quality threshold before it reaches users.

Ensure that your AI-generated text passage properly caters to its intended grade level before shipping.

## What you'll do

* Score a passage with the [Grade Level Appropriateness](https://docs.learningcommons.org/evaluators/text-complexity-evaluators/grade-level-appropriateness.md) evaluator
  * Treat a grade band above your target as a failed check
  * When it fails, use the [Vocabulary Complexity](https://docs.learningcommons.org/evaluators/text-complexity-evaluators/vocabulary-complexity.md) evaluator for the words and reasoning to send back to your model
* Ask your model to rewrite the passage, if needed
* Score the rewritten passage before it reaches a student

## What you'll need

* Node.js or Python
* [Google API key](https://aistudio.google.com/) ↗ (needed for Grade Level Appropriateness and Vocabulary Complexity)
* [OpenAI API key](https://platform.openai.com/) ↗ (needed for Vocabulary Complexity)
* Text passage and the grade band you want it to stay within

## Steps

**1. Install the SDK**

Install the package for the language your pipeline uses:

**TypeScript**

```shell TypeScript theme={null}
npm install @learning-commons/evaluators@latest
npm install @ai-sdk/google @ai-sdk/openai # Provider adaptors for the required API keys
```

**Python**

```shell Python theme={null}
pip install --upgrade learning-commons-evaluators
```

**2. Assess the grade level for your text passage**

> [!WARNING]
>
> Inputs must be de-identified. Do not submit students' Personally
> Identifiable Information (PII) or any regulated or sensitive personal
> information.

[Grade Level Appropriateness](https://docs.learningcommons.org/evaluators/text-complexity-evaluators/grade-level-appropriateness.md) takes a text passage and returns the grade band at which a student can read independently:

```typescript TypeScript theme={null}
import { GradeLevelAppropriatenessEvaluator } from "@learning-commons/evaluators";

const passage =
  "Honeybees forage for nectar, transport it to the colony, and convert it into honey through enzymatic processing inside the hive.";

const gradeLevel = new GradeLevelAppropriatenessEvaluator({
  googleApiKey: process.env.GOOGLE_API_KEY,
});

const { result } = await gradeLevel.evaluate({ text: passage });

console.log(result.grade_band); // "K-1" | "2-3" | "4-5" | "6-8" | "9-10" | "11-12"
console.log(result.reasoning);
```

```python Python theme={null}
import os

from learning_commons_evaluators import GradeLevelAppropriatenessEvaluator

passage = (
  "Honeybees forage for nectar, transport it to the colony, and "
  "convert it into honey through enzymatic processing inside the hive."
)

grade_level = GradeLevelAppropriatenessEvaluator(
  google_api_key=os.environ["GOOGLE_API_KEY"],
)

result = grade_level.evaluate_sync(text=passage).result

print(result.grade_band)  # "K-1" | "2-3" | "4-5" | "6-8" | "9-10" | "11-12"
print(result.reasoning)
```

**3. Determine whether the result meets your target**

Compare the returned grade band with your desired target. If the result is higher, it means the passage is too difficult for your audience and should not ship.

```typescript TypeScript theme={null}
const GRADE_BANDS = ["K-1", "2-3", "4-5", "6-8", "9-10", "11-12"] as const;
const TARGET_BAND = "4-5";

const aboveTarget =
  GRADE_BANDS.indexOf(result.grade_band) > GRADE_BANDS.indexOf(TARGET_BAND);

if (!aboveTarget) {
  console.log("Passes. Ship this version.");
} else {
  console.log(`Above ${TARGET_BAND}. Rewrite before it ships.`);
}
```

```python Python theme={null}
GRADE_BANDS = ["K-1", "2-3", "4-5", "6-8", "9-10", "11-12"]
TARGET_BAND = "4-5"

above_target = GRADE_BANDS.index(result.grade_band) > GRADE_BANDS.index(TARGET_BAND)

if not above_target:
  print("Passes. Ship this version.")
else:
  print(f"Above {TARGET_BAND}. Rewrite before it ships.")
```

**4. Rewrite the passage, if needed**

If the result grade band is too high, run [Vocabulary Complexity](https://docs.learningcommons.org/evaluators/text-complexity-evaluators/vocabulary-complexity.md) on the same passage.

Share the returned `tier_2_words`, `tier_3_words`, and `reasoning` with your model to help it rewrite the message for the target grade level.

```typescript TypeScript theme={null}
import { VocabularyComplexityEvaluator } from "@learning-commons/evaluators";

const vocabulary = new VocabularyComplexityEvaluator({
  googleApiKey: process.env.GOOGLE_API_KEY,
  openaiApiKey: process.env.OPENAI_API_KEY,
});

const { result: vocabularyResult } = await vocabulary.evaluate({
  text: passage,
  grade_level: "4",
});

const constraints = [
  "Rewrite the passage so a grade 4 student can read it independently.",
  `Do not use these Tier 3 words unless you define them: ${vocabularyResult.tier_3_words}`,
  `Also review these Tier 2 words: ${vocabularyResult.tier_2_words}`,
  `Why the vocabulary was scored this way: ${vocabularyResult.reasoning}`,
].join("\n\n");

console.log(constraints); // Send these constraints + original passage back to your model
```

```python Python theme={null}
from learning_commons_evaluators import VocabularyComplexityEvaluator

vocabulary = VocabularyComplexityEvaluator(
  google_api_key=os.environ["GOOGLE_API_KEY"],
  openai_api_key=os.environ["OPENAI_API_KEY"],
)

vocabulary_result = vocabulary.evaluate_sync(
  text=passage,
  grade_level=4,
).result

constraints = "\n\n".join([
  "Rewrite the passage so a grade 4 student can read it independently.",
  f"Do not use these Tier 3 words unless you define them: {vocabulary_result.tier_3_words}",
  f"Also review these Tier 2 words: {vocabulary_result.tier_2_words}",
  f"Why the vocabulary was scored this way: {vocabulary_result.reasoning}",
])

print(constraints) # Send these constraints + original passage back to your model
```

> [!TIP]
>
> Keep track of how many times your passage must be rewritten. A growing count is a sign the base prompt needs a change.
>
> Check out [Improve quality by refining prompts](https://docs.learningcommons.org/evaluators/getting-started/tutorials/improve-quality-by-refining-prompts.md) for more details on this use case.

**5. Score the rewrite before it ships**

Rerun [Grade Level Appropriateness](https://docs.learningcommons.org/evaluators/text-complexity-evaluators/grade-level-appropriateness.md) on your rewritten passage. Ship only if the result is at or below your target grade level.

```typescript TypeScript theme={null}
const revisedText =
  "Honeybees visit flowers to collect a sweet liquid called nectar. They carry it back to the hive and turn it into honey.";

const { result: revised } = await gradeLevel.evaluate({ text: revisedText });

const stillAboveTarget =
  GRADE_BANDS.indexOf(revised.grade_band) > GRADE_BANDS.indexOf(TARGET_BAND);

if (stillAboveTarget) {
  console.log("Still above the target band. Reconsider shipping this version.");
} else {
  console.log(`Passes at ${revised.grade_band}. Ship this version.`);
}
```

```python Python theme={null}
revised_text = (
  "Honeybees visit flowers to collect a sweet liquid called nectar. "
  "They carry it back to the hive and turn it into honey."
)

revised = grade_level.evaluate_sync(text=revised_text).result

still_above_target = GRADE_BANDS.index(revised.grade_band) > GRADE_BANDS.index(TARGET_BAND)

if still_above_target:
  print("Still above the target band. Reconsider shipping this version.")
else:
  print(f"Passes at {revised.grade_band}. Ship this version.")
```

## Related topics

> **[Resource Library - Vet content quality in production](https://learningcommons.org/resources/vet-content-quality-production/)**
>
> Score every output in your pipeline, then rewrite and re-check anything that
> misses the rubric.

> **[Quickstart](https://docs.learningcommons.org/evaluators/getting-started/quickstart.md)**
>
> Run an evaluator in the Evaluators playground, a Python notebook, or with
> the SDK.

> **[Tutorial: Monitor quality in production](https://docs.learningcommons.org/evaluators/getting-started/tutorials/monitor-quality-in-production.md)**
>
> Track grade band across a sample of outputs and catch a shift over time.

> **[Tutorial: Improve quality by refining prompts](https://docs.learningcommons.org/evaluators/getting-started/tutorials/improve-quality-by-refining-prompts.md)**
>
> Compare two prompt versions on grade band, vocabulary, and sentence
> structure.