Skip to main content
v0.4.0

​
What you’ll do

Evaluate a batch of text from a CSV file using all literacy evaluators. Results are output in both CSV and HTML format.

​
What you’ll need

  • Install the SDK globally
  • Create a CSV file with the text you want to evaluate
    • Must be 50 or fewer input rows (unless using the --bypass-row-limit option)
    • Must have text and grade columns
    • May include additional columns (will be preserved as-is in the output)
example.csv

​
Running the batch evaluator

Run the batch evaluator using npx from any directory:

​
Interactive prompts

If you omit a required input, the CLI prompts you interactively for the CSV file path, API keys, and/or output directory.
Before starting evaluations, the CLI always shows a confirm prompt. This intentional safety checkpoint helps prevent accidental expensive runs.

​
Options

Pass options to override the batch evaluator’s defaults:
When the run completes, the CLI prints the full path to the HTML report so you can open it directly.

​
Results

You’ll see a real-time display of the batch evaluator’s progress:
The batch evaluator will generate 2 files in your output directory:
results.csv
  • Spreadsheet-compatible format
  • Original CSV columns preserved
  • New CSV columns for each evaluator
    • {evaluator}_score
    • {evaluator}_reasoning
    • {evaluator}_status
results.html
  • Summary dashboard with grade-level distribution and text complexity charts
  • Scores and reasoning for each evaluator
If any evaluations fail (even after retries), only those rows will error out. The batch evaluator will skip those rows and then ultimately surface those failures in the results with an error status.

​
Graceful shutdown

If you press Ctrl+C during evaluation:
  • In-flight evaluations finish processing
  • Pending tasks are cancelled
  • Completed results are saved to results-partial.* files to preserve progress
If you press Ctrl+C twice to force quit immediately, you may lose in-flight results.

Quickstart

Install the SDK and run your first evaluator.

Configuration

Configure evaluators, including API keys and options.

Outputs

Understand evaluator output fields and types.

Error handling

Handle configuration, validation, and API errors.