> ## Documentation Index > Fetch the complete documentation index at: https://docs.learningcommons.org/llms.txt > Use this file to discover all available pages before exploring further. # Core concepts > Learn the fundamental concepts of evaluators including what they are, how they work, and key terminology for using evaluators effectively. ## How evaluators are designed | Concept | Definition | | :---------------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | |

[Evaluator](#evaluator)

| Evaluates AI-generated content by focusing on one [dimension](#dimension) of a [rubric](#rubric)
**Example:** The [Vocabulary evaluator](/evaluators/literacy-evaluators/vocabulary) focuses on one dimension (vocabulary) of the Qualitative Text Complexity (rubric). | |

[Dimension](#dimension)

| Describes one specific attribute or criteria of AI-generated content
**Example:** Evaluators can look at Vocabulary, Sentence Structure, and other dimensions of the text when evaluating its complexity. | |

[Rubric](#rubric)

| Provides a framework for evaluating a concept (e.g., Qualitative Text Complexity) through multiple [dimensions](#dimension)

Illustration showing a sample rubric structure used by evaluators.
**Example:** The Qualitative Text Complexity rubric looks at dimensions like Vocabulary and Sentence Structure to make a judgment on a text's complexity. | |

[Evaluators family](#evaluators-family)

| Collection of evaluators that evaluate [dimensions](#dimension) of one [rubric](#rubric)**Example:** The [Literacy evaluators family](/evaluators/literacy-evaluators/introduction) contains multiple evaluators that each assess one dimension of the Qualitative Text Complexity rubric. | ## How outputs are validated | Concept | Definition | | :---------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | |

[Evaluator dataset](#evaluator-dataset)

| [Dataset](/evaluators/dataset/introduction) rigorously annotated by human domain experts according to a [rubric](#rubric)

These annotated datasets train and inform our "LLM as a judge" assessments, making our evaluators more accurate with every iteration.
**Example:** We partnered with literacy experts to create the [Literacy dataset](/evaluators/dataset/literacy-dataset), which we also used to develop our [Literacy evaluators](/evaluators/literacy-evaluators/introduction). | |

[LLM as a judge](#llm-as-a-judge)

| Evaluation method that automates human-like reasoning at scale

Learning Commons uses this evaluation method to develop our evaluators. We prompt LLMs to use our expert-annotated [evaluator datasets](#evaluator-dataset) to validate our evaluators' outputs. | |

[Baseline prompt](#baseline-prompt)

| Simulates what an edtech developer might construct quickly without expert-annotated datasets, structured rubric alignment, or prompt optimization

Learning Commons uses baseline prompts measure how much our expert-annotated [evaluator datasets](/evaluators/dataset/introduction) improve our evaluators' performance. | |

[Accuracy](#accuracy)

| Degree to which an evaluator aligns with our [evaluator datasets](/evaluators/dataset/introduction), or how often an evaluator agrees with a human expert

Learn more about [How accuracy is measured](#how-accuracy-is-measured) and our recommended [use cases](#use-cases). | ## How accuracy is measured An evaluator's accuracy is calculated on every evaluation run per input (**single-run accuracy**). Because LLM outputs are probabilistic and inherently non-deterministic, accuracy results may vary across evaluation runs (even with the same inputs). To reduce your evaluators' variability and increase their reliability in production: 1. Run the evaluator multiple times on the same input 2. Treat each output as a vote 3. Select the majority result (for example, 3 runs with majority selection) ### Metrics Not all evaluators will return all accuracy metrics. | Accuracy metric | Definition | | :------------------------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | |

[Overall accuracy](#overall-accuracy)

| Percentage of predictions that exactly match the expert-annotated [evaluator dataset](#evaluator-dataset) Some disagreements between our evaluators and human experts are expected, especially in qualititative domains and borderline cases. | |

[Expert agreement rate](#expert-agreement-rate)

| Percentage of reviewed cases where experts agree with evaluator output | |

[Reasoning quality score](#reasoning-quality-score)

| Expert rating of explanation quality (often on a numeric scale) | |

[Baseline comparison accuracy](#baseline-comparison-accuracy)

| Performance relative to a minimal [baseline prompt](#baseline-prompt) | ### Use cases Use accuracy metrics to inform implementation decisions, not to replace judgment. | Recommended | Not recommended | | :----------------------------------------------- | :-------------------------------------------------------- | | Estimate expected agreement with expert judgment | Fully automate high-stakes decisions | | Set appropriate thresholds for downstream logic | Assume consistent performance outside the validated scope | | | Replace domain expertise where interpretation is critical |