GroundednessProEvaluator Class

Initialize a Groundedness Pro evaluator for determine if the response is grounded in the query and context.

If this evaluator is supplied to the evaluate function, the aggregated metric for the groundedness pro label will be "groundedness_pro_passing_rate".

Usage

   azure_ai_project = {
       "subscription_id": "<subscription_id>",
       "resource_group_name": "<resource_group_name>",
       "project_name": "<project_name>",
   }
   credential = DefaultAzureCredential()

   eval_fn = GroundednessProEvaluator(azure_ai_project, credential)
   result = eval_fn(query="What's the capital of France", response="Paris", context="Paris.")

Output format

   {
       "groundedness_pro_label": True,
       "reason": "'All Contents are grounded"
   }

Usage with conversation input

   azure_ai_project = {
       "subscription_id": "<subscription_id>",
       "resource_group_name": "<resource_group_name>",
       "project_name": "<project_name>",
   }
   credential = DefaultAzureCredential()

   eval_fn = GroundednessProEvaluator(azure_ai_project, credential)
   conversation = {
       "messages": [
           {"role": "user", "content": "What is the capital of France?"},
           {"role": "assistant", "content": "Paris.", "context": "Paris."}
           {"role": "user", "content": "What is the capital of Germany?"},
           {"role": "assistant", "content": "Berlin.", "context": "Berlin."}
       ]
   }
   result = eval_fn(conversation=conversation)

Output format

   {
       "groundedness_pro_label": 1.0,
       "evaluation_per_turn": {
           "groundedness_pro_label": [True, True],
           "groundedness_pro_reason": ["All contents are grounded", "All contents are grounded"]
       }
   }