Error handling

The SDK provides specific error types to help you handle different scenarios.

example.py

import time

from learning_commons_evaluators import (
  ConfigurationError,
  EvaluatorError,
  GradeLevelAppropriatenessEvaluator,
  RateLimitError,
)

try:
  evaluator = GradeLevelAppropriatenessEvaluator(google_api_key="your-google-key")
except ConfigurationError as error:
  raise SystemExit(error)

for attempt in range(3):
  try:
    result = evaluator.evaluate_sync(text="The cat's out of the bag now.")
    break
  except EvaluatorError as error:
    if not error.retryable or attempt == 2:
      raise
    wait_ms = error.retry_after_ms if isinstance(error, RateLimitError) else None
    time.sleep(wait_ms / 1000 if wait_ms else 2 ** attempt)

The SDK already retries internally according to max_retries: provider calls, Knowledge Graph requests, and immediate resamples when a model’s response fails its output schema. retryable tells you whether an error that still reached you is worth another attempt at your level.

Errors

All errors extend EvaluatorError and expose a boolean retryable flag.

Type	Description
`AuthenticationError`	External system rejected the credentials
`ConfigurationError`	Missing or invalid key, unknown provider or model, or malformed settings
`InputValidationError`	Invalid input (text too short, invalid grade, an image outside an evaluator’s limits, etc.), or a caller-supplied identifier a dependency rejects
`KnowledgeGraphError`	Catch-all for Knowledge Graph failures not intercepted by a more specific error (retryable if 5xx)
`LLMOutputProcessingError`	Model response failed parsing, normalization, or its output schema; retry immediately
`LLMProviderError`	Catch-all for LLM-provider failures not intercepted by a more specific error (retryable if 5xx)
`NetworkError`	Network connectivity issues
`RateLimitError`	Rate limit exceeded; wait and retry (`retry_after_ms`)
`RequestTimeoutError`	Request timed out
`StandardNotFoundError`	Standards code does not exist in the jurisdiction (subtype of `InputValidationError`)

EvaluatorError, EvaluationError, and DependencyError are abstract. The SDK always raises a concrete subclass.

`DependencyError`

Describes which external system failed:

Field	Type	Description
`dependency`	`"openai" \| "google" \| "anthropic" \| "knowledge-graph" \| "custom"`	Failed system
`status_code`	`int \| None`	HTTP status, when the dependency reported one
`request_id`	`str \| None`	Dependency’s request id, for support requests
`model`	`str \| None`	Model in play, for LLM dependencies

`InputValidationError`

Graphics evaluators run several checks on each image before any model call:

Check	Description
File	Image path is a regular file and can be read
Size	File size falls within the evaluator’s limits
Format	File is a PNG, JPEG, or WebP
Dimensions	File’s width, height, and pixel count fall within evaluator limits

An image that fails a check raises an InputValidationError, which includes a message that names the input and its position:

image_paths[0]: "chart.png" is 3000×2000 px; each edge must be 16 to 2560 px. Resize to 2560 px on the long edge before submitting.

Quickstart

Install the SDK and run your first evaluator.

Configuration

Configure evaluators, including API keys and options.

Evaluation

Understand evaluator output fields and types.

Understanding Evaluators

Getting started

Using Evaluators

Evaluators

Datasets

Resources

Errors

`DependencyError`

`InputValidationError`

Quickstart

Configuration

Evaluation

​Errors

​DependencyError

​InputValidationError

​Related topics

Quickstart

Configuration

Evaluation

Errors

`DependencyError`

`InputValidationError`

Related topics