> ## Documentation Index
> Fetch the complete documentation index at: https://docs.learningcommons.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Conventionality

> Reference documentation for the Conventionality literacy evaluator.

export const EarlyAccessCallout = ({children}) => <div className="eyebrow-callout not-prose rounded-xl border border-gray-200/80 p-5 dark:border-white/10" style={{
  marginBottom: "1rem",
  borderRadius: "4px"
}}>
    <div className="mb-3">
      <Badge color="green" size="md" icon="flask">
        Early access
      </Badge>
    </div>
    <div className="callout-body text-[15px] leading-relaxed text-gray-700 dark:text-gray-300">{children}</div>
    <style>{`.callout-body a { text-decoration: underline; text-decoration-color: #178251; }`}</style>
  </div>;

[Evaluator last updated March 20, 2026.](#evaluator-release-history)

<EarlyAccessCallout>
  This functionality is actively evolving. Changes may occur as we expand
  capabilities and improve accuracy and reliability. Email
  [support@learningcommons.org](mailto:support@learningcommons.org) ↗ with your
  feedback or issues.
</EarlyAccessCallout>

## Overview

The Conventionality evaluator assesses how directly a text communicates its meaning. It analyzes whether language is literal and explicit or relies on figurative, abstract, or implied meaning that requires interpretation.

## At a glance

|                      |                                                                                                                                                                                                             |
| :------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Input type**       | Informational text                                                                                                                                                                                          |
| **Supported grades** | 3–12                                                                                                                                                                                                        |
| **Rubric**           | [SAP](https://learnwithsap.org/) ↗'s [Qualitative Text Complexity Rubric for Informational Text](https://learnwithsap.b-cdn.net/app/uploads/2026/04/Qualitative-Text-Complexity-Rubric-Informational.pdf) ↗ |

The evaluator was built and validated using the model and temperature below (other configurations will produce different results and may have lower accuracy):

|                 |                        |
| :-------------- | :--------------------- |
| **Model used**  | gemini-3-flash-preview |
| **Temperature** | 0                      |

## Getting started

Follow the [Quickstart](/evaluators/getting-started/quickstart) to start using this evaluator:

| Access method             |                                                                                                                                                                                                                                                                                                                                                                     |
| :------------------------ | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Evaluators Playground** | [View in the Learning Commons Platform](https://platform.learningcommons.org/apps/evaluators/playground?utm_source=docs\&utm_medium=evaluators) ↗                                                                                                                                                                                                                   |
| **SDK**                   | [Python](https://github.com/learning-commons-org/evaluators/blob/main/sdks/python/src/learning_commons_evaluators/evaluators/conventionality.py?utm_source=docs\&utm_medium=evaluators) ↗ and [TypeScript](https://github.com/learning-commons-org/evaluators/blob/main/sdks/typescript/src/evaluators/conventionality.ts?utm_source=docs\&utm_medium=evaluators) ↗ |
| **Python notebook**       | [View in GitHub](https://github.com/learning-commons-org/evaluators/blob/main/evals/conventionality_evaluator.ipynb?utm_source=docs\&utm_medium=evaluators) ↗                                                                                                                                                                                                       |
| **Prompts**               | [View in GitHub](https://github.com/learning-commons-org/evaluators/tree/main/evals/prompts/conventionality/?utm_source=docs\&utm_medium=evaluators) ↗                                                                                                                                                                                                              |

## Inputs

| Input                  | Description                                             | Required |
| :--------------------- | :------------------------------------------------------ | :------- |
| **Target grade level** | Enables grade context evaluation                        | Yes      |
| **Text type**          | Informational text<br />Optional length 200-1,000 words | Yes      |

## Output

| Field                        | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| :--------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Complexity rating**        | <ul><li><strong>Slightly complex:</strong> Confirm generated text uses literal, grade-appropriate language<ul><li>Language is literal and explicit</li><li>Meaning is directly stated</li><li>Minimal use of abstract, symbolic, ironic, and/or figurative language</li><li>Almost entirely explicit, literal, and straight forward</li></ul></li><li><strong>Moderately complex:</strong> Check whether occasional figurative language fits instructional goals<ul><li>Mostly literal language with occasional figurative or implicit meaning</li><li>Occasional use of abstract, symbolic, ironic, and/or figurative language</li><li>Primarily explicit, literal and straightforward</li></ul></li><li><strong>Very complex:</strong> Plan scaffolding when figurative language is central to meaning<ul><li>Frequent figurative language or implied meaning requires interpretation</li><li>Frequent use of abstract, symbolic, ironic, and/or figurative language</li></ul></li><li><strong>Exceedingly complex:</strong> Flag texts that require heavy interpretive support before use<ul><li>Language relies heavily on abstraction, layered meaning, or sustained figurative expression</li><li>Pervasive use of abstract, symbolic, ironic, and/or figurative language</li></ul></li></ul> |
| **Reasoning**                | Explanation of the rating based on language features                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| **Conventionality features** | Specific language features driving complexity (for example, idioms, metaphors, irony, or implicit meaning)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| **Grade context**            | Comparison of conventionality demands with expectations for the provided grade                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| **Instructional insights**   | Suggestions for scaffolding or teaching unconventional language features<br /><br />**Example**: Use conventionality features, grade context, and reasoning together to identify which language features to pre-teach or clarify                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |

## Accuracy and validation

<Note>
  This evaluator is provided as Early access. Comprehensive accuracy measures
  are still evolving, and validation testing is ongoing.
</Note>

The evaluator was optimized using **35 annotated passages** and validated through expert review of additional samples from the [CLEAR Corpus](https://docs.google.com/spreadsheets/d/1sfsZhhP2umXXtmEP_NRErxLuwgN98TyH7LWOq3j07O0/edit) ↗:

| Metric                        | **Description**                                                                                                          | **Result**                            |
| :---------------------------- | :----------------------------------------------------------------------------------------------------------------------- | :------------------------------------ |
| **Complexity score accuracy** | How accurately the evaluator determines conventionality complexity compared to expert annotations.                       | 83% agreement with expert annotations |
| **Expert agreement**          | The percentage of evaluated examples where at least one expert agreed with the evaluator's rating during review testing. | 90% (9 of 10 examples approved)       |
| **Reasoning soundness**       | Expert rating of how well the evaluator's reasoning explains the complexity decision, scored on a 1–5 scale.             | Average 4.4 / 5                       |

## Evaluator release history

| Date           | Changed       |
| -------------- | ------------- |
| March 20, 2026 | First release |