RelevanceEvaluator Class

Initialize a relevance evaluator configured for a specific Azure OpenAI model.

Usage

   eval_fn = RelevanceEvaluator(model_config)
   result = eval_fn(
       query="What is the capital of Japan?",
       response="The capital of Japan is Tokyo.")

Output format

   {
       "relevance": 3.0,
       "gpt_relevance": 3.0,
       "relevance_reason": "The response is relevant to the query because it provides the correct answer.",
   }

Note: To align with our support of a diverse set of models, a key without the gpt_ prefix has been added. To maintain backwards compatibility, the old key with the gpt_ prefix is still be present in the output; however, it is recommended to use the new key moving forward as the old key will be deprecated in the future.