Configuration

Import and configure the evaluator of your choice.

example.py

from learning_commons_evaluators import (
  GoogleLLMProviderConfig,
  OpenAILLMProviderConfig,
  AnthropicLLMProviderConfig,
  GradeLevelAppropriatenessEvaluator,
  create_config,
)

# Create provider config
google_config = GoogleLLMProviderConfig(api_key="...")
openai_config = OpenAILLMProviderConfig(api_key="...")
anthropic_config = AnthropicLLMProviderConfig(api_key="...")

# Create evaluator config
# NOTE: Telemetry is not yet implemented in v0.2.0
config = create_config(
  google_llm_provider_config=google_config,
  telemetry_partner_id="your-learning-commons-api-key",
)

# Instantiate evaluator
evaluator = GradeLevelAppropriatenessEvaluator(config)

`create_config` methods

The SDK provides several methods to create a configuration object:

from learning_commons_evaluators import (
  create_config, # Collects telemetry (Not yet supported)
  create_config_no_telemetry, # Does not collect telemetry
  create_config_telemetry_with_full_input, # Collects telemetry with input text (Not yet supported)
)

Each method takes in a list of configuration options.

Options

Our evaluators are validated against a particular provider and model during development. Evaluators default to that same provider and model at runtime – for example, Grade Level Appropriateness will always use Google Gemini out of the box, because it was validated against Google Gemini during development.As a result, each evaluator requires specific API keys when being configured (e.g., google_llm_provider_config, openai_llm_provider_config, etc.).We recommend using the validated provider and model, but the SDK does allow you to override that default in your per-instance settings or per-call settings. Please note that your mileage may vary if you use a different model.

Field	Type	Description
`google_llm_provider_config`	`GoogleLLMProviderConfig`	Google API key, required for evaluators that default to Google models
`openai_llm_provider_config`	`OpenAILLMProviderConfig`	OpenAI API key, required for evaluators that default to OpenAI models
`anthropic_llm_provider_config`	`AnthropicLLMProviderConfig`	Anthropic API key, required for evaluators that default to Claude or when overriding a default model with Claude
`logger`	`Logger`	Custom logger for evaluator output. Can set logging verbosity level.
`telemetry_partner_id` Not yet supported	`string`	Your Learning Commons API key, used for authenticated and anonymous telemetry data collection. Required for `create_config` and `create_config_telemetry_with_full_input` only

Logging

Customize how your evaluator logs information.

Log level

Control logging verbosity:

# SDK uses Python's standard logging module
import logging

# By default, EvaluatorConfig uses the package logger,
# which propagates to the root logger once your app configures handlers

logging.basicConfig(level=logging.DEBUG)
logging.getLogger("learning_commons_evaluators").setLevel(logging.WARNING)

from learning_commons_evaluators import (
    create_config_no_telemetry,
    create_logger,
    create_silent_logger,
)

logger = create_logger(level=logging.DEBUG)

# Discards logs + does not send as telemetry data
config = create_config_no_telemetry(logger=create_silent_logger())

Custom logger

You can configure your evaluator with a custom logger:

import logging

from learning_commons_evaluators import (
  create_config,
)

# Route SDK logs through your own logger
my_logger = logging.getLogger("my_app.evaluators")
custom_logger_config = create_config(..., logger=my_logger)

Per-instance settings

Override the default for every call on a particular evaluator instance using default_evaluation_settings. Per-call settings will override per-instance settings for a single run.

from dataclasses import replace

from learning_commons_evaluators import GradeLevelAppropriateness

settings = GradeLevelAppropriateness.default_evaluation_settings.model_copy(deep=True)
settings.prompt_settings_step_gla_evaluation = replace(
  settings.prompt_settings_step_gla_evaluation,
  temperature=0.2,
  model="gemini-2.5-pro",
)

evaluator = GradeLevelAppropriateness(config, default_evaluation_settings=settings)

result = evaluator.evaluate_sync(input) # uses instance default

Per-call settings

Override models, temperatures, or other configurable evaluator settings for a single run using evaluation_settings:

from dataclasses import replace

from learning_commons_evaluators import GradeLevelAppropriateness

evaluator = GradeLevelAppropriateness(config)
settings = evaluator.default_evaluation_settings.model_copy(deep=True)
settings.prompt_settings_step_gla_evaluation = replace(
  settings.prompt_settings_step_gla_evaluation,
  temperature=0.2,
)

result = evaluator.evaluate_sync(input, evaluation_settings=settings)

When omitted, the evaluator uses a deep copy of its default settings.

Quickstart

Install the SDK and run your first evaluator.

Outputs

Understand evaluator output fields and types.

Error handling

Handle configuration, validation, and API errors.

Understanding evaluators

Getting started

SDK API reference

Literacy evaluators

Feedback evaluators

Standards evaluators

Datasets

Resources

`create_config` methods

Options

Logging

Log level

Custom logger

Per-instance settings

Per-call settings

Quickstart

Outputs

Error handling

​create_config methods

​Options

​Logging

​Log level

​Custom logger

​Per-instance settings

​Per-call settings

​Related topics

Quickstart

Outputs

Error handling

`create_config` methods

Options

Logging

Log level

Custom logger

Per-instance settings

Per-call settings

Related topics