QAEvaluator Class

Initialize a question-answer evaluator configured for a specific Azure OpenAI model.

Usage

   eval_fn = QAEvaluator(model_config)
   result = qa_eval(
       query="Tokyo is the capital of which country?",
       response="Japan",
       context="Tokyo is the capital of Japan.",
       ground_truth="Japan"
   )

Output format

   {
       "groundedness": 3.5,
       "relevance": 4.0,
       "coherence": 1.5,
       "fluency": 4.0,
       "similarity": 3.0,
       "gpt_groundedness": 3.5,
       "gpt_relevance": 4.0,
       "gpt_coherence": 1.5,
       "gpt_fluency": 4.0,
       "gpt_similarity": 3.0,
       "f1_score": 0.42
   }