Evaluator last updated March 20, 2026.
Overview
The Conventionality evaluator assesses how directly a text communicates its meaning. It analyzes whether language is literal and explicit or relies on figurative, abstract, or implied meaning that requires interpretation.
At a glance
| |
|---|
| Input type | Informational text |
| Supported grades | 3–12 |
| Rubric | SAP ↗‘s Qualitative Text Complexity Rubric for Informational Text ↗ |
| |
|---|
| Model used | gemini-3-flash-preview |
| Temperature | 0 |
Getting started
Follow the Quickstart to start using this evaluator:
| Input | Description | Required |
|---|
| Target grade level | Enables grade context evaluation | Yes |
| Text type | Informational text
Optional length 200-1,000 words | Yes |
Output
| Field | Description |
|---|
| Complexity rating | - Slightly complex: Confirm generated text uses literal, grade-appropriate language
- Language is literal and explicit
- Meaning is directly stated
- Minimal use of abstract, symbolic, ironic, and/or figurative language
- Almost entirely explicit, literal, and straight forward
- Moderately complex: Check whether occasional figurative language fits instructional goals
- Mostly literal language with occasional figurative or implicit meaning
- Occasional use of abstract, symbolic, ironic, and/or figurative language
- Primarily explicit, literal and straightforward
- Very complex: Plan scaffolding when figurative language is central to meaning
- Frequent figurative language or implied meaning requires interpretation
- Frequent use of abstract, symbolic, ironic, and/or figurative language
- Exceedingly complex: Flag texts that require heavy interpretive support before use
- Language relies heavily on abstraction, layered meaning, or sustained figurative expression
- Pervasive use of abstract, symbolic, ironic, and/or figurative language
|
| Reasoning | Explanation of the rating based on language features |
| Conventionality features | Specific language features driving complexity (for example, idioms, metaphors, irony, or implicit meaning) |
| Grade context | Comparison of conventionality demands with expectations for the provided grade |
| Instructional insights | Suggestions for scaffolding or teaching unconventional language features
Example: Use conventionality features, grade context, and reasoning together to identify which language features to pre-teach or clarify |
Accuracy and validation
This evaluator is provided as Early access. Comprehensive accuracy measures
are still evolving, and validation testing is ongoing.
| Metric | Description | Result |
|---|
| Complexity score accuracy | How accurately the evaluator determines conventionality complexity compared to expert annotations. | 83% agreement with expert annotations |
| Expert agreement | The percentage of evaluated examples where at least one expert agreed with the evaluator’s rating during review testing. | 90% (9 of 10 examples approved) |
| Reasoning soundness | Expert rating of how well the evaluator’s reasoning explains the complexity decision, scored on a 1–5 scale. | Average 4.4 / 5 |
Evaluator release history
| Date | Changed |
|---|
| March 20, 2026 | First release |
