Early Release

This evaluator reflects early-stage work. We’re continuously improving its accuracy and reliability.
The sentence structure evaluation has two major components:
  • sentence analysis and
  • assigning complexity
Both have a system and a user prompt. The sentence analysis component has Python code to provide supplemental sentence structure analysis features and to normalize statistics.

Sentence analysis

This instructs the evaluator to look at the passage sentence-by-sentence and create statistics to represent important sentence features. The evaluator takes the passage and some ground truth statistics, including:
  • Number of sentences
  • Words
  • Characters
  • Syllables in the passage
  • The passage’s Flesch-Kincaid score.
These statistics are calculated using the Python textstat library.

System prompt (analysis component)

You are an expert in grammar and literacy.

User prompt (analysis component)

# Task
I am going to give you a text, and I need you to look through the text sentence-by-sentence to perform a comprehensive grammatical analysis. Use the computational counts as a reference; they can be incorrect in ambiguous cases.

# Definitions
* Sentences: Count a complete grammatical unit ending in a terminal punctuation mark.
* Words: Count any sequence of characters separated by a space as one word. Treat hyphenated words (e.g., "state-of-the-art") and numbers (e.g., "2025") as single words.
* Independent Clauses: Clauses that can stand alone as a complete sentence.
* Subordinate Clauses: Clauses that are dependent on the main clause and cannot stand alone as a complete sentence.
* Simple Sentences: Sentences with one independent clause and no subordinate clauses.
* Compound Sentences: Sentences with two or more independent clauses and no subordinate clauses.
* Complex Sentences: Sentences with one independent clause and at least one subordinate clause.
* Compound-Complex Sentences: Sentences with two or more independent clauses and at least one subordinate clause.
* Other / Non-Canonical Sentences: Sentences that cannot be reliably classified as simple, compound, complex, or compound-complex (e.g., sentence fragments, run-ons, elliptical responses, headlines, imperatives lacking an explicit subject, or stylized dialogue tags).
* Subordinate Clauses: Clauses that are dependent on the main clause and cannot stand alone as a complete sentence.
* Embedded Clauses: Clauses that are nested within another clause.
* Prepositional Phrases: Phrases that begin with a preposition and end with a noun phrase.
* Participle Phrases: Phrases that begin with a participle and end with a noun phrase.
* Appositive Phrases: Phrases that rename or identify a noun phrase.
* Simple Transitions: Basic coordinating conjunctions and chronological adverbs. Examples: 'and', 'but', 'or', 'so', 'then', 'next', 'first'.
* Sophisticated Transitions: Conjunctive adverbs and phrases signaling logical relationships. Examples: 'however', 'therefore', 'consequently', 'as a result', 'for example', 'although'.
* One-Concept Sentence: A sentence with ZERO subordinate clauses AND ZERO transition words/phrases (neither simple nor sophisticated).
* Multi-Concept Sentence: Any sentence that has ≥1 subordinate clause OR ≥1 transition word/phrase (or both).

# Computational Counts
Use these as reference, your internal heuristics can be more reliable.
{ground_truth_counts}

# Text to Analyze
{text}

IMPORTANT: Your response should be a single JSON object with the following structure. Do not produce anything outside of the JSON object. 

{format_instructions}

Format instructions

This is a set of instructions for formatting the output in JSON.
The output should be formatted as a JSON instance that conforms to the JSON schema below.

As an example, for the schema {"properties": {"foo": {"title": "Foo", "description": "a list of strings", "type": "array", "items": {"type": "string"}}}, "required": ["foo"]}
the object {"foo": ["bar", "baz"]} is a well-formatted instance of the schema. The object {"properties": {"foo": ["bar", "baz"]}} is not well-formatted.

Here is the output schema:
```
{"properties": {"reasoning": {"description": "Your step-by-step reasoning as you analyze the text, with a line break between each sentence analyzed.", "title": "Reasoning", "type": "string"}, "num_sentences": {"description": "Total number of sentences in the text.", "title": "Num Sentences", "type": "integer"}, "num_words": {"description": "Total number of words in the text.", "title": "Num Words", "type": "integer"}, "flesch_kincaid_grade": {"description": "Flesch-Kincaid Grade Level number from Computational Counts, rounded to two decimal places.", "title": "Flesch Kincaid Grade", "type": "number"}, "num_simple_sentences": {"description": "Number of simple sentences in the text.", "title": "Num Simple Sentences", "type": "integer"}, "num_compound_sentences": {"description": "Number of compound sentences in the text.", "title": "Num Compound Sentences", "type": "integer"}, "num_complex_sentences": {"description": "Number of complex sentences in the text.", "title": "Num Complex Sentences", "type": "integer"}, "num_compound_complex_sentences": {"description": "Number of compound-complex sentences in the text.", "title": "Num Compound Complex Sentences", "type": "integer"}, "num_other_sentences": {"description": "Number of sentences that do not fit the four canonical types (e.g., fragments, run-ons, elliptical answers, headlines, or stylized dialogue tags).", "title": "Num Other Sentences", "type": "integer"}, "num_independent_clauses": {"description": "Total count of all independent clauses in the text.", "title": "Num Independent Clauses", "type": "integer"}, "num_subordinate_clauses": {"description": "Total count of all subordinate clauses in the text. A single sentence can have multiple.", "title": "Num Subordinate Clauses", "type": "integer"}, "num_total_clauses": {"description": "The sum of all independent and subordinate clauses in the text.", "title": "Num Total Clauses", "type": "integer"}, "num_sentences_with_subordinate": {"description": "Number of sentences that contain a subordinate clause.", "title": "Num Sentences With Subordinate", "type": "integer"}, "num_sentences_with_multiple_subordinates": {"description": "Number of sentences that contain two or more subordinate clauses.", "title": "Num Sentences With Multiple Subordinates", "type": "integer"}, "num_sentences_with_embedded_clauses": {"description": "Number of sentences with an embedded clause (a subordinate clause inside another clause).", "title": "Num Sentences With Embedded Clauses", "type": "integer"}, "num_prepositional_phrases": {"description": "Number of prepositional phrases in the text.", "title": "Num Prepositional Phrases", "type": "integer"}, "num_participle_phrases": {"description": "Number of participle phrases in the text.", "title": "Num Participle Phrases", "type": "integer"}, "num_appositive_phrases": {"description": "Number of appositive phrases in the text.", "title": "Num Appositive Phrases", "type": "integer"}, "num_simple_transitions": {"description": "Number of simple transitions in the text.", "title": "Num Simple Transitions", "type": "integer"}, "num_sophisticated_transitions": {"description": "Number of sophisticated transitions in the text.", "title": "Num Sophisticated Transitions", "type": "integer"}, "words_in_simple_sentences": {"description": "Total number of words in all sentences classified as simple.", "title": "Words In Simple Sentences", "type": "integer"}, "words_in_compound_sentences": {"description": "Total number of words in all sentences classified as compound.", "title": "Words In Compound Sentences", "type": "integer"}, "words_in_complex_sentences": {"description": "Total number of words in all sentences classified as complex.", "title": "Words In Complex Sentences", "type": "integer"}, "words_in_compound_complex_sentences": {"description": "Total number of words in all sentences classified as compound-complex.", "title": "Words In Compound Complex Sentences", "type": "integer"}, "words_in_other_sentences": {"description": "Total number of words in all sentences classified as other.", "title": "Words In Other Sentences", "type": "integer"}, "sentence_word_counts": {"description": "A list containing the word count of each individual sentence in the order they appear.", "items": {"type": "integer"}, "title": "Sentence Word Counts", "type": "array"}, "num_one_concept_sentences": {"description": "Number of sentences with a single main idea: no subordinate clause and no transition word/phrase.", "title": "Num One Concept Sentences", "type": "integer"}, "num_multi_concept_sentences": {"description": "Number of sentences with multiple main ideas: either a subordinate clause or a transition word/phrase or both.", "title": "Num Multi Concept Sentences", "type": "integer"}, "num_cleft_sentences": {"description": "Number of sentences with cleft constructions (e.g., \\"It was X that...\\", \\"What X did was...\\").", "title": "Num Cleft Sentences", "type": "integer"}, "max_clauses_in_any_sentence": {"description": "Max number of clauses (independent + subordinate) found in a single sentence.", "title": "Max Clauses In Any Sentence", "type": "integer"}}, "required": ["reasoning", "num_sentences", "num_words", "flesch_kincaid_grade", "num_simple_sentences", "num_compound_sentences", "num_complex_sentences", "num_compound_complex_sentences", "num_other_sentences", "num_independent_clauses", "num_subordinate_clauses", "num_total_clauses", "num_sentences_with_subordinate", "num_sentences_with_multiple_subordinates", "num_sentences_with_embedded_clauses", "num_prepositional_phrases", "num_participle_phrases", "num_appositive_phrases", "num_simple_transitions", "num_sophisticated_transitions", "words_in_simple_sentences", "words_in_compound_sentences", "words_in_complex_sentences", "words_in_compound_complex_sentences", "words_in_other_sentences", "sentence_word_counts", "num_one_concept_sentences", "num_multi_concept_sentences", "num_cleft_sentences", "max_clauses_in_any_sentence"]}
```

Assigning complexity

This instructs the evaluator to assign an overall complexity rating to the text. Note that there are different prompts used for Grade 3 and Grade 4.

System prompt

You are an expert in grammar and literacy, and understand K-12 and SCASS text complexity rubric

Overall user prompt

Your task is to perform a text complexity analysis for a Grade {grade} student. You will be given a text excerpt and a set of quantitative sentence-level statistics for that text.

You must integrate both the qualitative aspects of the text and the quantitative statistics to make your final judgment. Do not rely on the numbers alone.

1.  Read the TEXT EXCERPT to understand its topic, conceptual load, and overall structure.
2.  Review the TEXT STATISTICS as a guide for complexity level.
3.  Synthesize your findings in your reasoning. Explain how the structure (qualitative) interact with the text statistics (quantitative) to determine the complexity. For example, a text with simple sentences might still be complex if the topic is very dense or abstract.

Your final answer must be one of ["Slightly Complex," "Moderately Complex," "Very Complex", "Exceedingly Complex"].

# GRADE {grade} RUBRIC
{rubric}

# TEXT EXCERPT
{excerpt}

# TEXT STATISTICS
{sentence_features}

# OUTPUT FORMAT
{format_instructions}
NOTE: {sentence_features} is the set of statistics generated via the sentence labeling prompt.

Grade 3 rubric for prompt

**Instructions for Analysis:** First, evaluate if the text meets the criteria for "Slightly Complex" or "Exceedingly Complex". If it does not fit into these categories, then decide between "Moderately Complex" and "Very Complex".

**Slightly Complex:**
* **Description:** The text consists of simple, straightforward language and sentence structures.
* **Statistical Guidelines:** The text is likely "Slightly Complex" if it meets at least TWO of the following criteria:
    * **Sentence Type:** Primarily simple sentences. (`percent_simple_sentences` is typically > 60%).
    * **Sentence Length:** Short sentences. (`avg_sentence_length` is typically < 12 words).
    * **Subordination:** Very low use of clauses. (`percent_sentences_with_subordinate` is typically < 25%).

**Moderately Complex:**
* **Description:** The text shows a mix of simple and more complex sentences, introducing some variety in structure without being overly demanding.
* **Statistical Guidelines:** If the text is not "Slightly Complex", consider "Moderately Complex" if it generally aligns with these ranges:
    * **Sentence Type:** A balanced mix of sentence types. (`percent_simple_sentences` is typically between 40% and 60%).
    * **Sentence Length:** Medium length sentences. (`avg_sentence_length` is typically between 12 and 16 words).
    * **Subordination:** A moderate use of clauses. (`percent_sentences_with_subordinate` is typically between 25% and 45%).

**Very Complex:**
* **Description:** The text features more elaborate sentences with multiple clauses and ideas, requiring more effort from the reader to parse. This is often the default category for grade-level text that isn't simple or exceptionally difficult.
* **Statistical Guidelines:** If the text is more complex than "Moderately" but does not meet the "Exceedingly" criteria, it is likely "Very Complex". Key indicators include:
    * **Sentence Type:** Complex structures are common. (`percent_simple_sentences` is a minority, typically < 40%).
    * **Sentence Length:** Longer sentences are frequent. (`avg_sentence_length` is typically between 16 and 19 words).
    * **Subordination:** Subordinate clauses are a key feature. (`percent_sentences_with_subordinate` is typically > 45%).

**Exceedingly Complex:**
* **Description:** The text is dense with very long, intricate sentences and a high degree of subordination, making it exceptionally challenging for this grade level.
* **Statistical Guidelines:** The text is "Exceedingly Complex" if it shows an extreme combination of sentence length and structural density. It should meet at least **TWO** of the following criteria, including at least **ONE** from the "Structural Density" group.
    * **Structural Density Indicators:**
        * High Subordination: `percent_sentences_with_subordinate` is extensive (typically > 50%).
        * Multiple Subordinates: `percent_sentences_with_multiple_subordinates` is consistently present (typically > 12%).
        * High Syntactic Complexity: `percent_compound_complex_sentences` is significant (typically > 15%).
    * **Length and Density Indicators:**
        * Extreme Sentence Length: `avg_sentence_length` is very long (typically > 19 words).
        * Low Simplicity: `percent_simple_sentences` is very low (typically < 30%).
        * Concentrated Length: `percent_very_long_sentences` is notable (typically > 10%).

Grade 4 rubric for prompt

**Instructions for Analysis:** First, evaluate if the text meets the criteria for "Slightly Complex" or "Exceedingly Complex". If it does not fit into these categories, then decide between "Moderately Complex" and "Very Complex".

**Slightly Complex:**
* **Description:** The text uses clear, direct language with basic sentence structures appropriate for developing readers.
* **Statistical Guidelines:** The text is likely "Slightly Complex" if it meets at least TWO of the following criteria:
    * **Sentence Type:** Dominated by simple sentences. (`percent_simple_sentences` is typically > 55%).
    * **Sentence Length:** Short to medium sentences. (`avg_sentence_length` is typically < 13 words).
    * **Subordination:** Infrequent use of clauses. (`percent_sentences_with_subordinate` is typically < 30%).

**Moderately Complex:**
* **Description:** The text contains a variety of sentence structures, including compound and complex sentences, but remains accessible.
* **Statistical Guidelines:** If the text is not "Slightly Complex", consider "Moderately Complex" if it generally aligns with these ranges:
    * **Sentence Type:** A healthy mix of sentence types. (`percent_simple_sentences` is typically between 40% and 55%).
    * **Sentence Length:** Medium length sentences. (`avg_sentence_length` is typically between 13 and 17 words).
    * **Subordination:** A moderate number of clauses. (`percent_sentences_with_subordinate` is typically between 30% and 50%).

**Very Complex:**
* **Description:** The text is characterized by longer sentences and the regular use of dependent clauses, requiring readers to track multiple ideas. This is the default for challenging, on-grade-level texts.
* **Statistical Guidelines:** If the text is more complex than "Moderately" but does not meet the "Exceedingly" criteria, it is likely "Very Complex". Key indicators include:
    * **Sentence Type:** Simple sentences are a clear minority. (`percent_simple_sentences` is typically < 40%).
    * **Sentence Length:** Sentences are consistently long. (`avg_sentence_length` is typically between 17 and 22 words).
    * **Subordination:** Subordination is a major feature. (`percent_sentences_with_subordinate` is typically > 50%).
    * **Multiple Subordination:** Sentences with multiple clauses appear more often. (`percent_sentences_with_multiple_subordinates` is typically > 8%).

**Exceedingly Complex:**
* **Description:** The text's structure is highly sophisticated and dense, marked by extensive use of embedded clauses and long, flowing sentences that are well above grade-level expectations.
* **Statistical Guidelines:** A text is "Exceedingly Complex" if its structure is highly sophisticated and dense. It should meet at least **TWO** of the following criteria, including at least **ONE** from the "Structural Density" group.
    * **Structural Density Indicators:**
        * High Subordination: `percent_sentences_with_subordinate` is very high (typically > 60%).
        * Multiple Subordinates: `percent_sentences_with_multiple_subordinates` is high and consistent (typically > 15%).
        * High Syntactic Complexity: `percent_compound_complex_sentences` is a notable feature (typically > 20%).
    * **Length and Density Indicators:**
        * Extreme Sentence Length: `avg_sentence_length` is exceptionally long (typically > 22 words).
        * Low Simplicity: `percent_simple_sentences` is very low (typically < 25%).
        * Concentrated Length: `percent_very_long_sentences` is significant (typically > 15%).

Formatting instructions

This is a set of instructions for formatting the output in JSON.
 
The output should be formatted as a JSON instance that conforms to the JSON schema below.

# This is Pydantic JsonOutputParser text
As an example, for the schema {"properties": {"foo": {"title": "Foo", "description": "a list of strings", "type": "array", "items": {"type": "string"}}}, "required": ["foo"]}
the object {"foo": ["bar", "baz"]} is a well-formatted instance of the schema. The object {"properties": {"foo": ["bar", "baz"]}} is not well-formatted.

Here is the output schema:
```
{"properties": {"reasoning": {"description": "detailed reasoning that is pedagogically approriate, and helpful for K-12 educators", "title": "Reasoning", "type": "string"}, "answer": {"description": "the final complexity category, which must be one of ['Slightly Complex', 'Moderately Complex', 'Very Complex', 'Exceedingly Complex'].", "title": "Answer", "type": "string"}}, "required": ["reasoning", "answer"]}
```