Conventionality

Evaluator last updated March 20, 2026.

Overview

The Conventionality evaluator assesses how directly a text communicates its meaning. It analyzes whether language is literal and explicit or relies on figurative, abstract, or implied meaning that requires interpretation.

At a glance


Input type	Informational text
Supported grades	3–12
Rubric	SAP ↗‘s Qualitative Text Complexity Rubric for Informational Text ↗

The evaluator was built and validated using the model and temperature below (other configurations will produce different results and may have lower accuracy):


Model used	gemini-3-flash-preview
Temperature	0

Getting started

Follow the Quickstart to start using this evaluator:

Access method
Evaluators Playground	View in the Learning Commons Platform ↗
SDK	Python ↗ and TypeScript ↗
Python notebook	View in GitHub ↗
Prompts	View in GitHub ↗

Inputs

Input	Description	Required
Target grade level	Enables grade context evaluation	Yes
Text type	Informational text Optional length 200-1,000 words	Yes

Output

Field	Description
Complexity rating	Slightly complex: Confirm generated text uses literal, grade-appropriate language Language is literal and explicit Meaning is directly stated Minimal use of abstract, symbolic, ironic, and/or figurative language Almost entirely explicit, literal, and straight forward Moderately complex: Check whether occasional figurative language fits instructional goals Mostly literal language with occasional figurative or implicit meaning Occasional use of abstract, symbolic, ironic, and/or figurative language Primarily explicit, literal and straightforward Very complex: Plan scaffolding when figurative language is central to meaning Frequent figurative language or implied meaning requires interpretation Frequent use of abstract, symbolic, ironic, and/or figurative language Exceedingly complex: Flag texts that require heavy interpretive support before use Language relies heavily on abstraction, layered meaning, or sustained figurative expression Pervasive use of abstract, symbolic, ironic, and/or figurative language
Reasoning	Explanation of the rating based on language features
Conventionality features	Specific language features driving complexity (for example, idioms, metaphors, irony, or implicit meaning)
Grade context	Comparison of conventionality demands with expectations for the provided grade
Instructional insights	Suggestions for scaffolding or teaching unconventional language features Example: Use conventionality features, grade context, and reasoning together to identify which language features to pre-teach or clarify

Accuracy and validation

This evaluator is provided as Early access. Comprehensive accuracy measures are still evolving, and validation testing is ongoing.

The evaluator was optimized using 35 annotated passages and validated through expert review of additional samples from the CLEAR Corpus ↗:

Metric	Description	Result
Complexity score accuracy	How accurately the evaluator determines conventionality complexity compared to expert annotations.	83% agreement with expert annotations
Expert agreement	The percentage of evaluated examples where at least one expert agreed with the evaluator’s rating during review testing.	90% (9 of 10 examples approved)
Reasoning soundness	Expert rating of how well the evaluator’s reasoning explains the complexity decision, scored on a 1–5 scale.	Average 4.4 / 5

Evaluator release history

Date	Changed
March 20, 2026	First release

Understanding evaluators

Getting started

SDK API Reference

Literacy evaluators

Feedback evaluators

Standards evaluators

Datasets

Resources

Overview

At a glance

Getting started

Inputs

Output

Accuracy and validation

Evaluator release history

​Overview

​At a glance

​Getting started

​Inputs

​Output

​Accuracy and validation

​Evaluator release history

Overview

At a glance

Getting started

Inputs

Output

Accuracy and validation

Evaluator release history