Quickstart

What you’ll do

Select an evaluator to run from one of the following evaluator families:

For this quickstart, we will run the Grade Level Appropriateness evaluator (from the Text Complexity evaluator family) to evaluate some text for a given grade level.

Access method	When to use
Evaluators playground ↗	For a quick demo of how evaluators work
SDK	To integrate into your TypeScript or Python project
Python notebook	For quick prototyping

What you’ll need

Text you want to evaluate
Grade level of the intended audience
API keys required by the evaluators you want to use Not required for Evaluators playground

Required API keys

By default, each evaluator uses a provider and model that we have tested for reliability for that task. As a result, different evaluators require different API keys.

Evaluator family	Required API key(s)
Text Complexity evaluators See exceptions below:	Google
Vocabulary Complexity	Google and OpenAI
Sentence Structure	OpenAI
Graphics evaluators	Google
Feedback evaluators	OpenAI
Academic Standards evaluators	Anthropic and Learning Commons
Durable Skills evaluators	Anthropic

You can override our evaluators’ defaults with any Google, OpenAI, or Anthropic model. When overriding with an Anthropic model, an Anthropic API key is required.

Steps

Evaluators playground
SDK
Python notebooks

The Evaluators playground ↗ is the easiest way to see our evaluators in action.

Select evaluator family

Choose the evaluators you want to run on your input (e.g. Text Complexity evaluators or Feedback evaluators).

Provide inputs

Text Complexity evaluators: Choose the grade level of your intended audience, and copy and paste the content you want to evaluate.Feedback evaluators: Copy and paste the student text and the feedback text you want to evaluate.

Run the evaluators

Click Evaluate to run our Text Complexity evaluators or Feedback evaluators on your input.

View results

Expand each evaluator’s results to see its score and reasoning.

The Evaluators playground is primarily a demo. Check out the SDK or Python notebooks when implementing evaluators into your automated processes.

Install the SDK

Run evaluators from your project by installing the SDK of your choice.


Python	GitHub ↗	Quickstart
TypeScript	GitHub ↗	Quickstart

Import and run evaluators

Import evaluators to start evaluating text in your project:

import { GradeLevelAppropriatenessEvaluator } from "@learning-commons/evaluators";

const evaluator = new GradeLevelAppropriatenessEvaluator({
  googleApiKey: process.env.GOOGLE_API_KEY,
});

const { result } = await evaluator.evaluate({
  text: "The cat's out of the bag now.",
});

console.log(result.grade_band); // e.g. 2-3

from learning_commons_evaluators import GradeLevelAppropriatenessEvaluator

evaluator = GradeLevelAppropriatenessEvaluator(
  google_api_key="your-google-key",
)

result = evaluator.evaluate_sync(
  text="The cat's out of the bag now.",
).result

print(result.grade_band)  # e.g. 2-3

See the Python SDK quickstart and TypeScript SDK quickstart for more implementation details.

The Python interpreter powers our evaluators. All downloadable examples ↗ and tutorials are provided as Python snippets.

Mac/Linux
Windows

Install Python 3.11 or newer. To verify your version of Python:

python3 --version

Clone the examples

The examples, their rubrics, and their dependency list all live in the evaluators repository ↗. Cloning it gives you notebooks you can open and run directly.

git clone https://github.com/learning-commons-org/evaluators.git
cd evaluators

Run the remaining steps from inside that folder.

Create a virtual environment

Creating an isolated environment prevents conflicts between Python packages used in this project and others on your system:

python3 -m venv .venv

source .venv/bin/activate # Activates environment

Remember to activate the virtual environment for each new shell session.

Install dependencies

Install all required packages listed in requirements.txt ↗:

pip install -r evals/requirements.txt

Set your API keys

Set your API keys as environment variables in your shell session:

export OPENAI_API_KEY="sk-your-key-here"
export GOOGLE_API_KEY="your-key-here"

Run an evaluator

Start a Jupyter Notebook in your web browser (usually at http://localhost:8888):

jupyter lab

Every evaluator ships with a runnable notebook under evals/. Open one from the Jupyter Lab file browser and run its cells, for example:

evals/text-complexity/ela-reading/grade-level-appropriateness/example_notebook.ipynb

Each notebook sits beside that evaluator’s prompts, schemas, and fixtures, so you can read what it sends and change it in place.To start from scratch instead, click the Notebook: Python 3 (ipykernel) tile in the Jupyter Lab launcher, or select File > New > Notebook > Python 3 (ipykernel) in the menu, and copy in the code from any example.

You can also use an IDE with Python and Jupyter Notebook support (e.g. VS Code with Microsoft’s Python and Jupyter extensions).

Install Python 3.11 or newer. To verify your version of Python:

py --version

Clone the examples

The examples, their rubrics, and their dependency list all live in the evaluators repository ↗. Cloning it gives you notebooks you can open and run directly. This step needs Git ↗, which Windows does not include by default:

git clone https://github.com/learning-commons-org/evaluators.git
cd evaluators

Run the remaining steps from inside that folder.

Create a virtual environment

In Command Prompt:

py -m venv .venv
.venv\Scripts\activate

Or in PowerShell:

py -m venv .venv

.venv\Scripts\Activate.ps1

Remember to activate the virtual environment for each new shell session.

Install dependencies

Install all required packages listed in requirements.txt ↗:

pip install -r evals/requirements.txt

Set your API keys

Set your API keys as environment variables in Command Prompt:

set "OPENAI_API_KEY=sk-your-key-here"
set "GOOGLE_API_KEY=your-key-here"

Or in PowerShell:

$env:OPENAI_API_KEY="sk-your-key-here"
$env:GOOGLE_API_KEY="your-key-here"

Run an evaluator

Start a Jupyter Notebook in your web browser (usually at http://localhost:8888):

jupyter lab

Every evaluator ships with a runnable notebook under evals/. Open one from the Jupyter Lab file browser and run its cells, for example:

evals/text-complexity/ela-reading/grade-level-appropriateness/example_notebook.ipynb

You can also use an IDE with Python and Jupyter Notebook support (e.g. VS Code with Microsoft’s Python and Jupyter extensions).

Core concepts

Learn how dimensions, rubrics, and accuracy shape evaluator results.

Python SDK

Configure evaluators, interpret outputs, and handle errors in Python.

TypeScript SDK

Configure evaluators, interpret outputs, and handle errors in TypeScript.

Understanding Evaluators

Getting started

Using Evaluators

Evaluators

Datasets

Resources

What you’ll do

What you’ll need

Required API keys

Steps

Core concepts

Python SDK

TypeScript SDK

​What you’ll do

​What you’ll need

​Required API keys

​Steps

​Related topics

Core concepts

Python SDK

TypeScript SDK

What you’ll do

What you’ll need

Required API keys

Steps

Related topics