Introduction

​

How Evaluators work

In the following example, an evaluator checks a 4th grade science passage for multiple dimensions of text complexity, returning a rating with reasoning.

Video transcript

Say your product generates a 4th grade science text meant to stretch students a little, but not too much. It reads well. But do its vocabulary, sentences, and structure give the stretch you want?The evaluators check AI-generated content against expert-built rubrics, so you can determine if it’s what your audience needs. Just give the passage and target grade to the vocabulary complexity, sentence structure, and organizational structure evaluators through the TypeScript or Python SDK.The evaluator uses its rubric’s rating criteria to evaluate the text. Every evaluator is built and tested against examples education experts have already rated.This text comes back as having moderately complex sentence and organizational structure, which is the stretch your team decided on. But its vocabulary is very complex, more than you wanted. The reasoning points to a few words.Use the reasoning to revise your prompt, regenerate, and check again. Run it in your pipeline before users see the output, whether that’s one or 1,000 runs, or let them run it themselves. The evaluator gives you the signal, but you decide what’s right.These evaluators are part of the Text Complexity family. They, and all of our other evaluators, are designed to fit with the evaluators you already run. Other families cover feedback on student work, standards alignment, and durable skills like critical thinking. Mix and match them, like ensuring a standards-aligned math worksheet has grade-level-appropriate word problems. We’re continually adding evaluators across different educational contexts.Ready to jump in? Check out the Quickstart.

​

What Evaluators do

Evaluators assess the quality of AI-generated educational content by measuring specific dimensions of text and graphics and identifying areas for improvement. Evaluators can help you reliably assess your LLM outputs and build evidence-based tools that reinforce student learning.

​

How to access Evaluators

​

Our approach

Learning Commons collaborates closely with pedagogical experts to define, test, and build our evaluators. We follow a research-informed process to develop evaluators that are firmly anchored in learning science:

• We build alongside experts in learning science and rubric development (e.g., Student Achievement Partners ↗, CAST ↗, and Achievement Network (ANet) ↗).
• We translate expert insight into ground-truth datasets that reflect real teaching and learning principles.
• We develop, validate, and ship software that evaluates content the way an expert would.

​

Related topics