FluencyEvaluator Class

Initialize a fluency evaluator configured for a specific Azure OpenAI model.

Usage

   eval_fn = FluencyEvaluator(model_config)
   result = eval_fn(response="The capital of Japan is Tokyo.")

Output format

   {
       "fluency": 4.0,
       "gpt_fluency": 4.0,
   }

Note: To align with our support of a diverse set of models, a key without the gpt_ prefix has been added. To maintain backwards compatibility, the old key with the gpt_ prefix is still be present in the output; however, it is recommended to use the new key moving forward as the old key will be deprecated in the future.