Skip to main content
To use an evaluator, you need to:
  1. Set up your environment. (below)
  2. Download examples.
  3. Work through the tutorial.
  4. Build it into your app or process.

​
Requirements

We rely on the Python interpreter to power the evaluators. All examples and tutorials are provided as Python code snippets.

​
Steps

  • Mac/Linux set up
  • Windows set up
You’ll need Python 3.10 or newer. To verify your version of Python, run the following code in the terminal.
python3 --version

​
STEP 1: Create a virtual environment

Creating an isolated environment is a best practice that prevents conflicts between Python packages used in this project and others on your system.
python3 -m venv .venv

source .venv/bin/activate
Remember to activate the virtual environment for each new shell session when working with Evaluators.

​
STEP 2: Install dependencies

The list of required packages is provided in the requirements.txt file.
pip install -r evals/requirements.txt

​
STEP 3: Set your API keys

We are using both OpenAI and Google Geminifor different evaluators. You need API keys from both platforms:Set the keys as environment variables in your shell session.
export OPENAI_API_KEY="sk-your-key-here"
export GOOGLE_API_KEY="your-key-here"
</Tab>
<Tab title="Windows set up">
You’ll need Python 3.10 or newer. To verify your version of Python, run the following code in the terminal:

```shell
python --version

​
STEP 4: Run an evaluator

You are now ready to run the evaluator examples, like the Sentence Structure Evaluator. We recommend using a Jupyter Notebook for interactive exploration.
  1. Start Jupyter Notebook.
jupyter lab
Jupyter will open in your web browser (usually at http://localhost:8888).
  1. Create a new notebook by clicking the Notebook: Python 3 (ipykernel) tile in the Jupyter Lab launcher. Alternatively, navigate to the menu and select File > New > Notebook > Python 3 (ipykernel).
  2. You can now copy the code from our Python examples into the cells of your new notebook to run an evaluator. If you prefer using an IDE with Python and Jupyter Notebook support, such as VS Code with Microsoft’s Python and Jupyter extensions, please refer to Microsoft’s instructions for installation and configuration.