Quickstart

What you’ll do

Run our Literacy evaluators on some text to evaluate it for a given grade level.

Method	When to use
Evaluators Playground	For a quick demo of how evaluators work
SDK	To integrate into your TypeScript or Python project
Python notebook	For quick prototyping

What you’ll need

Text you want to evaluate
Grade level of the intended audience
API keys required by the evaluators you want to use Not required for Evaluators Playground

Required API keys

By default, each evaluator uses a provider and model that we have tested for reliability for that task. As a result, different evaluators require different API keys.

Evaluator	Required API key(s)
Literacy evaluators (except for Vocabulary and Sentence Structure)	Google
Vocabulary	Google and OpenAI
Sentence Structure	OpenAI
Feedback evaluators	OpenAI
Standards evaluators	Anthropic

You can override our evaluators’ defaults with any Google, OpenAI, or Anthropic model. When overriding with an Anthropic model, an Anthropic API key is required.

Steps

Evaluators Playground
SDK
Python notebooks

The Evaluators Playground ↗ is the easiest way to see our evaluators in action.

Select evaluator family

Choose the evaluators you want to run on your input (e.g. Literacy or Feedback).

Provide inputs

Literacy evaluators: Choose the grade level of your intended audience, and copy and paste the content you want to evaluate.Feedback evaluators: Copy and paste the student text and the feedback text you want to evaluate.

Run the evaluators

Click Evaluate to run our Literacy or Feedback evaluators on your input.

View results

Expand each evaluator’s results to see its score and reasoning.

The Evaluators Playground is primarily a demo. Check out the SDK or Python notebooks when implementing evaluators into your automated processes.

Install the SDK

Run evaluators from your project by installing the SDK of your choice.

Language	Installation	Latest version
TypeScript	GitHub ↗
Python	GitHub ↗

Contact us ↗ to request additional SDK language support. You can also sign up on the Learning Commons Platform ↗ for updates on availability.

Import and run evaluators

Import evaluators to start evaluating text in your project:

import { GradeLevelAppropriatenessEvaluator } from "@learning-commons/evaluators";

const evaluator = new GradeLevelAppropriatenessEvaluator({
  googleApiKey: process.env.GOOGLE_API_KEY,
});

const text = "The cat's out of the bag now.";

const result = await evaluator.evaluate(text);
console.log(result.score); // 4-5

import logging
from learning_commons_evaluators import (
    GradeLevelAppropriateness,
    GradeLevelAppropriatenessEvaluationInput,
    GoogleLLMProviderConfig,
    create_config_no_telemetry,
)

# Logger
logging.basicConfig(level=logging.INFO)

# Config with Google API key
config = create_config_no_telemetry(
  google_llm_provider_config=GoogleLLMProviderConfig(api_key="your-google-key"),
)

# Instantiate GradeLevelAppropriatenessEvaluator with created config
evaluator = GradeLevelAppropriatenessEvaluator(config)

# Evaluate some text for grade level appropriateness
result = evaluator.evaluate_sync(
  GradeLevelAppropriatenessEvaluationInput(text="The cat's out of the bag now.")
)

print(result.answer.label) # e.g. "Moderately complex"
print(result.explanation.summary) # Reasoning for the score

See the SDK API Reference for more implementation details.

The Python interpreter powers our evaluators. All downloadable examples ↗ and tutorials are provided as Python snippets.

Mac/Linux
Windows

Install Python 3.10 or newer. To verify your version of Python:

python3 --version

Create a virtual environment

Creating an isolated environment prevents conflicts between Python packages used in this project and others on your system:

python3 -m venv .venv

source .venv/bin/activate # Activates environment

Remember to activate the virtual environment for each new shell session.

Install dependencies

Install all required packages listed in requirements.txt ↗:

pip install -r evals/requirements.txt

Set your API keys

Set your API keys as environment variables in your shell session:

export OPENAI_API_KEY="sk-your-key-here"
export GOOGLE_API_KEY="your-key-here"

Run an evaluator

Start a Jupyter Notebook in your web browser (usually at http://localhost:8888):

jupyter lab

Create a new notebook by clicking the Notebook: Python 3 (ipykernel) tile in the Jupyter Lab launcher. Alternatively, select File > New > Notebook > Python 3 (ipykernel) in the menu.Run an evaluator by copying the code from our examples ↗ into the cells of your notebook.

You can also use an IDE with Python and Jupyter Notebook support (e.g. VS Code with Microsoft’s Python and Jupyter extensions).

Install Python 3.10 or newer. To verify your version of Python:

python3 --version

Create a virtual environment

In Command Prompt:

python -m venv .venv
.venv\Scripts\activate

Or in PowerShell:

python -m venv .venv

.venv\Scripts\Activate.ps1

Remember to activate the virtual environment for each new shell session.

Install dependencies

Install all required packages listed in requirements.txt ↗:

pip install -r evals/requirements.txt

Set your API keys

Set your API keys as environment variables in Command Prompt:

export OPENAI_API_KEY="sk-your-key-here"
export GOOGLE_API_KEY="your-key-here"

Or in PowerShell:

$env:OPENAI_API_KEY="sk-your-key-here"
$env:GOOGLE_API_KEY="your-key-here"

Run an evaluator

Start a Jupyter Notebook in your web browser (usually at http://localhost:8888):

jupyter lab

You can also use an IDE with Python and Jupyter Notebook support (e.g. VS Code with Microsoft’s Python and Jupyter extensions).

Core concepts

Learn how dimensions, rubrics, and accuracy shape evaluator results.

SDK API Reference

Configure evaluators, interpret outputs, and handle errors in your project.

Understanding evaluators

Getting started

SDK API Reference

Literacy evaluators

Feedback evaluators

Standards evaluators

Datasets

Resources

What you’ll do

What you’ll need

Required API keys

Steps

Core concepts

SDK API Reference

​What you’ll do

​What you’ll need

​Required API keys

​Steps

​Related topics

Core concepts

SDK API Reference

What you’ll do

What you’ll need

Required API keys

Steps

Related topics