Configuration

Import and configure the evaluator of your choice.

example.py

from learning_commons_evaluators import (
  EvaluatorConfig,
  GradeLevelAppropriatenessEvaluator,
)

# Pass a config object...
config = EvaluatorConfig(google_api_key="your-google-key")
evaluator = GradeLevelAppropriatenessEvaluator(config)

# ...or pass the same fields as keyword arguments
evaluator = GradeLevelAppropriatenessEvaluator(google_api_key="your-google-key")

Credentials are always passed in explicitly. The SDK never reads them from environment variables.

Options

Our evaluators are validated against a particular provider and model during development. Evaluators default to that same provider and model at runtime – for example, Grade Level Appropriateness will always use Google Gemini out of the box, because it was validated against Google Gemini during development. As a result, each evaluator requires specific API keys when being configured (e.g., google_api_key, openai_api_key, etc.).We recommend using the validated provider and model, but you can override that default with the model_override option.

Field	Type	Description
`google_api_key`	`str \| None`	Google API key, required for evaluators that default to Google models
`openai_api_key`	`str \| None`	OpenAI API key, required for evaluators that default to OpenAI models
`anthropic_api_key`	`str \| None`	Anthropic API key, required for evaluators that default to Claude or when `model_override` is set to `Provider.ANTHROPIC`
`learning_commons_api_key` v1.0.0	`str \| None`	Learning Commons API key ↗, required for all evaluators that call the Learning Commons REST API (e.g., Academic Standards evaluators) For telemetry, set `learning_commons_api_key` on the `telemetry` option instead.
`model_override`	`ModelOverride \| None` ↗	Overrides the evaluator’s default provider and model. Both `provider` and `model` are required. See Model override
`max_retries`	`int`	Default: `2`. Maximum retry attempts for failed evaluator calls. Total attempts are `1 + max_retries`. `0` disables retrying
`telemetry`	`bool \|` `TelemetryOptions` ↗	Default: `True`. Configuration for collecting telemetry. Set `learning_commons_api_key` on this option to attribute events to your Learning Commons user; omit the key for anonymous events.

Evaluators are validated and tested against their default models. Results with other models (using model_override) may vary.

Model override

model_override overrides the evaluator’s default provider and model. Both provider and model are required.

from learning_commons_evaluators import (
  EvaluatorConfig,
  GradeLevelAppropriatenessEvaluator,
  ModelOverride,
  Provider,
)

config = EvaluatorConfig(
  anthropic_api_key="your-anthropic-key",
  model_override=ModelOverride(provider=Provider.ANTHROPIC, model="claude-haiku-4-5-20251001"),
)

evaluator = GradeLevelAppropriatenessEvaluator(config)

Provider is a closed set: Provider.OPENAI, Provider.GOOGLE, and Provider.ANTHROPIC. An override replaces the evaluator’s own LLM key requirement with the override provider’s. The example above needs anthropic_api_key, not google_api_key. It does not skip credentials for a non-LLM service the evaluator still calls. A model id the provider rejects surfaces as a ConfigurationError when the provider answers.

Logging

Customize how your evaluator logs information. The SDK uses Python’s standard logging module, attaches a NullHandler to its own logger, and never calls logging.basicConfig() or configures the root logger. Each evaluator logs to a child of learning_commons_evaluators named for its registry id — for example, learning_commons_evaluators.text_complexity.ela_reading.grade_level_appropriateness.

Coming from the TypeScript SDK? There, logger and logLevel are constructor options. In Python you configure the standard library logger instead.

Log level

Control logging verbosity:

import logging

from learning_commons_evaluators import SDK_LOGGER_NAME

# Your application configures handlers; SDK records propagate to the root
logging.basicConfig(level=logging.DEBUG)

# Quiet the SDK while the rest of your app stays at DEBUG
logging.getLogger(SDK_LOGGER_NAME).setLevel(logging.WARNING)

# Or silence it entirely: the SDK's own NullHandler absorbs the records
logging.getLogger(SDK_LOGGER_NAME).propagate = False

Custom logger

Instead of injecting a logger, point the SDK’s logger at your handlers:

import logging

from learning_commons_evaluators import create_logger

# Pick one: every evaluator in the process...
create_logger(level=logging.INFO)

# ...a single evaluator, named by its registry id...
create_logger("text_complexity.ela_reading.grade_level_appropriateness", level=logging.DEBUG)

# ...or your own handler and format, in place of the default StreamHandler
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter("[%(levelname)s] %(name)s: %(message)s"))
create_logger(handler=handler, level=logging.INFO)

create_logger attaches a StreamHandler only when the target logger has none of its own, but a handler you pass is always added.

Telemetry

We collect limited usage and performance telemetry by default. This may include performance metrics (latency, token usage), technical metadata (such as SDK version and evaluator type) and related diagnostic information. This telemetry helps us improve evaluator quality, identify edge cases, and optimize performance. Telemetry never carries the text being evaluated. You can disable telemetry collection through the configuration options.

What you’ll need

While telemetry data collection is not required, we recommend enabling it so that we can better support your team’s use cases.

Telemetry is on by default. Events are anonymous unless you set learning_commons_api_key on the telemetry option. To attribute events to your Learning Commons user, generate an API key ↗ and include it in your evaluator’s configuration options:

from learning_commons_evaluators import EvaluatorConfig, TelemetryOptions

# Shorthand: off
config = EvaluatorConfig(google_api_key="...", telemetry=False)

# Granular: on, and attributed to your Learning Commons account
config = EvaluatorConfig(
  google_api_key="...",
  telemetry=TelemetryOptions(
    enabled=True,
    learning_commons_api_key="your-learning-commons-api-key",
  ),
)

We don’t collect your API keys or any user identifiers through the SDK.

What we collect

Field	Description
`timestamp`	ISO 8601 timestamp when evaluation started
`sdk_version`	SDK version (e.g., `"0.1.0"`)
`evaluator_type`	Which evaluator ran (registry id, e.g., `"text_complexity.ela_reading.vocabulary_complexity"`)
`grade`	Grade level evaluated. Empty for an evaluator that takes none. The wire field is `grade`, not `grade_level`
`status`	Evaluation outcome (`"success"` or `"error"`)
`error_code`	Error type if status is `"error"` (the exception class name, e.g., `"ConfigurationError"`)
`latency_ms`	Total evaluation time in milliseconds
`text_length_chars`	Length of input text in characters
`provider`	LLM provider(s) used (e.g., `"openai:gpt-4o"`, `"google:gemini-2.5-pro"`)
`model_override`	`true` when the caller supplied a `model_override`. Omitted otherwise
`token_usage`	Total tokens consumed (input and output)
`metadata.stage_details`	Per-stage breakdown for multi-stage evaluators (optional)

When telemetry is disabled, nothing is sent.

Quickstart

Install the SDK and run your first evaluator.

Evaluation

Understand evaluator output fields and types.

Error handling

Handle configuration, validation, and API errors.

Understanding evaluators

Getting started

SDK API reference

Text Complexity evaluators

Feedback evaluators

Academic standards evaluators

Durable skills evaluators

Datasets

Resources

Options

Model override

Logging

Log level

Custom logger

Telemetry

What you’ll need

What we collect

Quickstart

Evaluation

Error handling

​Options

​Model override

​Logging

​Log level

​Custom logger

​Telemetry

​What you’ll need

​What we collect

​Related topics

Quickstart

Evaluation

Error handling

Options

Model override

Logging

Log level

Custom logger

Telemetry

What you’ll need

What we collect

Related topics