example.py
The SDK already retries internally according to
max_retries: provider calls, Knowledge Graph requests, and immediate resamples when a model’s response fails its output schema. retryable tells you whether an error that still reached you is worth another attempt at your level.Errors
All errors extendEvaluatorError and expose a boolean retryable flag.
EvaluatorError, EvaluationError, and DependencyError are abstract. The SDK always raises a concrete subclass.
Dependency errors
Which external system failed is data on the error. EveryDependencyError carries:
Related topics
Quickstart
Install the SDK and run your first evaluator.
Configuration
Configure evaluators, including API keys and options.
Evaluation
Understand evaluator output fields and types.