Evaluation

Once you’ve configured your evaluator, you can start evaluating text inputs for grade level appropriateness:

example.py

from learning_commons_evaluators import (
  GradeLevelAppropriatenessEvaluator,
  GradeLevelAppropriatenessInput,
)

evaluator = GradeLevelAppropriatenessEvaluator(google_api_key="your-google-key")

result = evaluator.evaluate_sync(
  GradeLevelAppropriatenessInput(text="The cat's out of the bag now.")
)

Inputs

Inputs must be de-identified. Do not submit students' Personally Identifiable Information (PII) or any regulated or sensitive personal information.

Each evaluator requires different input fields when invoking evaluate().

Text Complexity evaluators

Grade Level Appropriateness
Others

Field	Type	Description
`text`	`string`	Informational text to evaluate (10-10,000 characters)

Field	Type	Description
`text`	`string`	Informational text to evaluate (10-10,000 characters)
`grade_level`	`int`	Target student grade (3–12)

from learning_commons_evaluators import (
  VocabularyComplexityEvaluator,
  VocabularyComplexityInput,
)

evaluator = VocabularyComplexityEvaluator(
  openai_api_key="your-openai-key",
  google_api_key="your-google-key",
)

result = evaluator.evaluate_sync(
  VocabularyComplexityInput(text="The cat's out of the bag now.", grade_level=5)
)

Graphics evaluators

v1.1.0 See InputValidationError to debug instances where an image fails a check before the model call.

Graphics Accuracy
Graphics Complexity

Field	Type	Description
`image_paths`	`list[str]`	Local path to the image to check, as a one-item list. PNG, JPEG, or WebP, 64 bytes to 5 MB, with width and height each 16–2,560 pixels
`claim`	`str`	What the image should show: a statement about the image, or a question with its expected answer. 1–12,000 characters

from learning_commons_evaluators import (
  GraphicsAccuracyEvaluator,
  compose_graphics_accuracy_claim, # Produces the exact wording the evaluator was tested with
)

evaluator = GraphicsAccuracyEvaluator(google_api_key="your-google-key")

result = evaluator.evaluate_sync(
  image_paths=["figures/apples.png"],
  claim=compose_graphics_accuracy_claim(
    "How many apples do all the trees have together?", "9"
  ),
)

Field	Type	Description
`text`	`str`	Informational text that the graphics accompany. 1–10,000 characters
`grade_level`	`int`	Target student grade, 3–12. Passed as an integer
`image_paths`	`list[str]`	Local paths to the passage’s graphics, one per graphic, in reading order. 1–5 images. PNG, JPEG, or WebP, 64 bytes to 2.5 MB, with width and height each 16–8,000 pixels
`figure_labels`	`str \| None`	Optional. Comma-separated labels in the same order as `image_paths` (e.g., `"Figure 1, Figure 2"`). Defaults to `Image 1`, `Image 2`, and so on

from learning_commons_evaluators import GraphicsComplexityEvaluator

evaluator = GraphicsComplexityEvaluator(google_api_key="your-google-key")

result = evaluator.evaluate_sync(
  text="Beekeepers in the United States report losing hives at rates they have not seen before.",
  grade_level=5,
  image_paths=["figures/hive-losses.png"],
  figure_labels="Figure 1",
)

Feedback evaluators

Field	Type	Description
`student_text`	`string`	Student’s written response. 1–10,000 characters
`feedback_text`	`string`	Teacher or AI feedback to evaluate. 1–10,000 characters

Academic Standards evaluators

The Academic Standards evaluators have 2 entry points, depending on how the standard is named:

evaluate() - By the standard’s CASE UUID
evaluate_by_code() - By the standard’s code

evaluate()
evaluate_by_code()

Field	Type	Description
`question`	`string`	Math assessment question to evaluate. 1–10,000 characters
`case_identifier_uuid`	`string`	CASE Network UUID of the math standard — the Knowledge Graph’s `caseIdentifierUUID`

from learning_commons_evaluators import MathStandardsAlignmentEvaluator

evaluator = MathStandardsAlignmentEvaluator(
  anthropic_api_key="your-anthropic-key",
  learning_commons_api_key="your-learning-commons-key",
)

result = evaluator.evaluate_sync(
  question="What is 3/4 + 1/8?",
  case_identifier_uuid="6ba25656-d7cc-11e8-824f-0242ac160002",
)

Field	Type	Description
`question`	`string`	Math assessment question to evaluate. 1–10,000 characters
`statement_code`	`string`	Math standard code, spelled as its jurisdiction spells it (e.g., `"3.MD.C.7.d"`, `"K.CC.A.1"`). 1–50 characters
`jurisdiction`	`Jurisdiction` ↗	Optional. Whose framework the code belongs to (`JurisdictionENUM`). Defaults to `Jurisdiction.MULTI_STATE`, which is
`grade_level`	`string`	Optional, `"K"` through `"12"`. Only consulted when a code names more than one standard, which happens when a framework reuses codes across courses

import asyncio

from learning_commons_evaluators import MathStandardsAlignmentEvaluator

evaluator = MathStandardsAlignmentEvaluator(
  anthropic_api_key="your-anthropic-key",
  learning_commons_api_key="your-learning-commons-key",
)

result = asyncio.run(
  evaluator.evaluate_by_code(
    question="What is 3/4 + 1/8?",
    statement_code="5.NF.A.1",
  )
)

Output

All evaluators return the following fields:

Field	Type	Description
`evaluator`	`string`	Registry ID for the evaluator used
`result`	Unique to each evaluator See “Output” for each evaluator (e.g., for Grade Level Appropriateness)	Detailed internal analysis data specific to each evaluator
`metadata`	`EvaluationMetadata` ↗	Includes model used to generate the evaluation, total evaluation processing time in milliseconds, input tokens, and output tokens

Quickstart

Install the SDK and run your first evaluator.

Configuration

Configure evaluators, including API keys and options.

Error handling

Handle configuration, validation, and API errors.

Understanding evaluators

Getting started

SDK API reference

Text Complexity evaluators

Graphics evaluators

Feedback evaluators

Academic standards evaluators

Durable skills evaluators

Datasets

Resources

Inputs

Text Complexity evaluators

Graphics evaluators

Feedback evaluators

Academic Standards evaluators

Output

Quickstart

Configuration

Error handling

​Inputs

​Text Complexity evaluators

​Graphics evaluators

​Feedback evaluators

​Academic Standards evaluators

​Output

​Related topics

Quickstart

Configuration

Error handling

Inputs

Text Complexity evaluators

Graphics evaluators

Feedback evaluators

Academic Standards evaluators

Output

Related topics