> ## Documentation Index
> Fetch the complete documentation index at: https://docs.learningcommons.org/llms.txt
> Use this file to discover all available pages before exploring further.

# About this evaluator

> Reference documentation for the Subject Matter Knowledge Evaluator.

export const EarlyAccessCallout = ({children}) => <div className="eyebrow-callout not-prose rounded-xl border border-gray-200/80 p-5 dark:border-white/10" style={{
  marginBottom: "1rem",
  borderRadius: "4px"
}}>
    <div className="mb-3">
      <Badge color="green" size="md" icon="flask">
        Early access
      </Badge>
    </div>
    <div className="callout-body text-[15px] leading-relaxed text-gray-700 dark:text-gray-300">{children}</div>
    <style>{`.callout-body a { text-decoration: underline; text-decoration-color: #178251; }`}</style>
  </div>;

[Evaluator last updated March 13, 2025.](#evaluator-release-history)

<EarlyAccessCallout>
  This functionality is actively evolving. Changes may occur as we expand capabilities and improve accuracy and reliability. Email [support@learningcommons.org](mailto:support@learningcommons.org) ↗ with your feedback or issues.
</EarlyAccessCallout>

## At a glance

|                      |                    |
| :------------------- | :----------------- |
| **Input type**       | Informational text |
| **Supported grades** | 3-12               |

The Subject Matter Knowledge Evaluator assesses the subject-matter complexity of reading passages using SAP's [Qualitative Text Complexity rubric (SAP)](https://achievethecore.org/file/823/file-823.pdf) ↗. It:

* Evaluates how much prior knowledge a text relies on.
* Lists key concepts in the text.
* Evaluates whether students have likely encountered these concepts before.

This evaluator was refined through repeated review by literacy experts so that its complexity ratings closely match expert judgment using qualitative rubrics.

## Model and prompt

For instructions on running the evaluator, see [Running an evaluator](/evaluators/using-evaluators/running-evaluators).

|                 |                                                                                                             |   |
| :-------------- | ----------------------------------------------------------------------------------------------------------- | - |
| **Model used**  | gemini-3-flash-preview                                                                                      |   |
| **Temperature** | 0                                                                                                           |   |
| **Prompts**     | [View prompts](https://github.com/learning-commons-org/evaluators/blob/main/evals/prompts/smk_prompts.py) ↗ |   |
| **Notebook**    | [View notebook](https://github.com/learning-commons-org/evaluators/blob/main/evals/smk_evaluator.ipynb) ↗   |   |

<Note>
  Other configurations will produce different results and may have lower accuracy.
</Note>

## Inputs

| Requirement            | Supported                                               | Required |
| ---------------------- | ------------------------------------------------------- | -------- |
| **Target grade level** | Enables grade context evaluation                        | Yes      |
| **Text type**          | Informational text<br />Optimal length: 200-1,000 words | Yes      |

## Output

| Field                       | Description                                                           |
| --------------------------- | --------------------------------------------------------------------- |
| Complexity rating           | Complexity level                                                      |
| Reasoning                   | Synthesized explanation of the decision                               |
| Identified topics           | List of topics in the text                                            |
| Curriculum check            | Classification of topics as general or specialized                    |
| Assumptions and scaffolding | Analysis of assumed prior knowledge vs. explained content             |
| Friction analysis           | Analysis of the gap between concrete description and abstract meaning |

## Interpreting results

This evaluator returns one of the following ratings, along with reasoning for you to use to determine your best course of action.

| Rating              | Meaning                                                                                                                                   |
| ------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| Slightly complex    | The text relies on everyday, practical knowledge and presents simple, concrete ideas.                                                     |
| Moderately complex  | The text relies on common practical knowledge and some discipline-specific content, with a mix of simple and more abstract ideas.         |
| Very complex        | The text relies on moderate discipline-specific or theoretical knowledge and mixes recognizable ideas with challenging abstract concepts. |
| Exceedingly complex | The text relies on extensive discipline-specific or theoretical knowledge and presents a range of challenging abstract concepts.          |

## Accuracy and validation

<Note>
  This evaluator is provided as Early access. \
  Comprehensive accuracy measures are not yet available. Validation testing is ongoing.
</Note>

We assessed performance against an expert-annotated dataset of texts, rated by 3 experts, rather than traditional static benchmark metrics. For more information, see [Accuracy](/evaluators/using-evaluators/accuracy).

| Metric                                                                                                                                                | Result                                                                                                                          |
| ----------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| <Tooltip tip="How accurately the evaluator determines text complexity compared to the expert annotated dataset. ">Complexity score accuracy</Tooltip> | 87% based on agreement with 3 experts assessing \~30 texts                                                                      |
| <Tooltip tip="The agreement rate between the evaluator and experts on a sample of 10 evaluated texts.">Expert agreement rate</Tooltip>                | 100%                                                                                                                            |
| Dataset source                                                                                                                                        | [CLEAR Corpus](https://www.commonlit.org/blog/introducing-the-clear-corpus-an-open-dataset-to-advance-research-28ff8cfea84a/) ↗ |

## Evaluator release history

| Date           | Changed        |
| -------------- | -------------- |
| March 13, 2026 | First release. |