Skip to main content

​
At a glance

Input typeInformational text
Supported grades3-12
Most recent versionv 1.3.0
The Subject Matter Knowledge Evaluator assesses the subject-matter complexity of reading passages using SAP’s Qualitative Text Complexity rubric (SAP). It:
  • Evaluates how much prior knowledge a text relies on.
  • Lists key concepts in the text.
  • Evaluates whether students have likely encountered these concepts before.
This evaluator was refined through repeated review by literacy experts so that its complexity ratings closely match expert judgment using qualitative rubrics.

​
Model and prompt

For instructions on running the evaluator, see Running an evaluator.
Model usedgemini-3-flash-preview
Temperature0
PromptsView prompts
NotebookView notebook
Other configurations will produce different results and may have lower accuracy.

​
Inputs

RequirementSupportedRequired
Target grade levelEnables grade context evaluationYes
Text typeInformational text
Optimal length: 200-1,000 words		Yes

​
Output

FieldDescription
Complexity ratingComplexity level
ReasoningSynthesized explanation of the decision
Identified topicsList of topics in the text
Curriculum checkClassification of topics as general or specialized
Assumptions and scaffoldingAnalysis of assumed prior knowledge vs. explained content
Friction analysisAnalysis of the gap between concrete description and abstract meaning

​
Interpreting results

This evaluator returns one of the following ratings, along with reasoning for you to use to determine your best course of action.
RatingMeaning
Slightly complexThe text relies on everyday, practical knowledge and presents simple, concrete ideas.
Moderately complexThe text relies on common practical knowledge and some discipline-specific content, with a mix of simple and more abstract ideas.
Very complexThe text relies on moderate discipline-specific or theoretical knowledge and mixes recognizable ideas with challenging abstract concepts.
Exceedingly complexThe text relies on extensive discipline-specific or theoretical knowledge and presents a range of challenging abstract concepts.

​
Accuracy and validation

This evaluator is provided as Early access.
Comprehensive accuracy measures are not yet available. Validation testing is ongoing.
We assessed performance against an expert-annotated dataset of texts, rated by 3 experts, rather than traditional static benchmark metrics. For more information, see Accuracy.
MetricResult
87% based on agreement with 3 experts assessing ~30 texts
100%
Dataset sourceCLEAR Corpus

​
Version history