CoherenceEvaluator Class

Initialize a coherence evaluator configured for a specific Azure OpenAI model.

Usage

   eval_fn = CoherenceEvaluator(model_config)
   result = eval_fn(
       query="What is the capital of Japan?",
       response="The capital of Japan is Tokyo.")

Output format

   {
       "coherence": 1.0,
       "gpt_coherence": 1.0,
   }

Note: To align with our support of a diverse set of models, a key without the gpt_ prefix has been added. To maintain backwards compatibility, the old key with the gpt_ prefix is still be present in the output; however, it is recommended to use the new key moving forward as the old key will be deprecated in the future.