Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.learningcommons.org/llms.txt

Use this file to discover all available pages before exploring further.

Import and configure the evaluator of your choice.
example.py
from learning_commons_evaluators import (
  GoogleLLMProviderConfig,
  OpenAILLMProviderConfig,
  AnthropicLLMProviderConfig,
  ConventionalityEvaluator,
  create_config,
)

# Create provider config
google_config = GoogleLLMProviderConfig(api_key="...")
openai_config = OpenAILLMProviderConfig(api_key="...")
anthropic_config = AnthropicLLMProviderConfig(api_key="...")

# Create evaluator config
# NOTE: Telemetry is not yet implemented in v0.1.0
config = create_config(
  google_llm_provider_config=google_config,
  telemetry_partner_id="your-learning-commons-api-key",
)

# Instantiate evaluator
evaluator = ConventionalityEvaluator(config)

​
Options

All evaluators accept some base configuration options.
Get the Required API keys for the evaluators you want to use.
Python provides several ways to create a configuration object: 
from learning_commons_evaluators import (
  create_config, # Collects telemetry
  create_config_no_telemetry, # Does not collect telemetry
  create_config_telemetry_with_full_input, # Collects telemetry with input text
)
FieldTypeDescription
google_llm_provider_configGoogleLLMProviderConfigGoogle API key, required for evaluators that default to Google models
openai_llm_provider_configOpenAILLMProviderConfigOpenAI API key, required for evaluators that default to OpenAI models
anthropic_llm_provider_configAnthropicLLMProviderConfigAnthropic API key, required for evaluators that default to Claude or when overriding a default model with Claude
loggerLoggerCustom logger for evaluator output. Can set logging verbosity level.
telemetry_partner_idstringYour Learning Commons API key, used for authenticated and anonymous telemetry data collection. Required for create_config and create_config_telemetry_with_full_input only

​
Logging

Customize how your evaluator logs information.

​
Log level

Control logging verbosity: 
# SDK uses Python's standard logging module
import logging

# By default, EvaluatorConfig uses the package logger,
# which propagates to the root logger once your app configures handlers

logging.basicConfig(level=logging.DEBUG)
logging.getLogger("learning_commons_evaluators").setLevel(logging.WARNING)

from learning_commons_evaluators import (
    create_config_no_telemetry,
    create_logger,
    create_silent_logger,
)

logger = create_logger(level=logging.DEBUG)

# Discards logs + does not send as telemetry data
config = create_config_no_telemetry(logger=create_silent_logger())

​
Custom logger

You can configure your evaluator with a custom logger: 
import logging

from learning_commons_evaluators import (
  create_config,
)

# Route SDK logs through your own logger
my_logger = logging.getLogger("my_app.evaluators")
custom_logger_config = create_config(..., logger=my_logger)