False positives when extracting math formulas

"fine-tune pre-trained models on your specific dataset to improve their performance in detecting and recognizing symbols accurately."

(How) can I fine-tune the pre-trained models provided by Microsoft like "prebuilt-layout" or the model of the formula feature add-on?
Do you have any resources or how-to pages for this?

Hello Clemens Fiedler,

To fine-tune pre-trained models provided by Microsoft, such as the "prebuilt-layout" or the formula feature add-on, you can utilize the Azure AI Foundry platform. Fine-tuning allows you to customize these models on your specific dataset, which can improve their performance in detecting and recognizing symbols accurately.

Here are some steps to fine-tune a model:

1. Choose a model that supports your task.
2. Prepare and upload your training data.
3. (Optional) Prepare and upload validation data.
4. (Optional) Configure task parameters.
5. Train your model.
6. Review metrics and evaluate the model's performance.
7. Use your fine-tuned model.

It's important to provide high-quality training data, ideally hundreds or thousands of examples, to achieve the best results.

Kindly refer below link: https://learn.microsoft.com/en-us/azure/ai-foundry/concepts/fine-tuning-overview

Thank You.

Hello Clemens Fiedler,

Thank you for your follow-up and feedback.

Sorry for misunderstanding me. Prebuilt models you referred to cannot be fine-tuned.

Instead of, misleading Information that Azure AI Foundry is not a valid Microsoft service, and the linked page that is broken. Also, the incorrect claim that prebuilt models like prebuilt-layout cannot be fine-tuned (Microsoft explicitly states prebuilt models are static) and none-practical guidance by @Saideep Anchuri

The best practices solution is to combine preprocessing, > post-processing, > custom models, and language filtering.

Refer to the following links:

• https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/concept-custom
• https://learn.microsoft.com/en-us/azure/ai-services/language-service/overview
• https://docs.python.org/3/library/re.html

Follow the steps here to solve your issue:

1. At preprocessing, Filter Non-Formula Patterns by using regex to exclude common false positives before sending text to Azure:

   import re
   def filter_non_formulas(text):
       # Remove bracketed ellipses [...]
       text = re.sub(r'\[\.\.\.\]', '', text)
       # Remove phone numbers (e.g., +1 123-456-789)
       text = re.sub(r'\+\d[\d\- ]+', '', text)
       return text

https://docs.python.org/3/library/re.html

1. Post-Processing, you will Flag Low-Confidence Formulas by using Azure’s confidence scores to discard weak formula predictions (e.g., formulas with only brackets/dots):

      formulas = [result for result in analysis_results if result.confidence > 0.8]
       
   

   https://learn.microsoft.com/en-us/python/api/azure-ai-documentintelligence/azure.ai.documentintelligence.models.documentformula?view=azure-python
2. Then, train a custom formula model using Document Intelligence Studio to prioritize LaTeX-like structures and exclude non-technical text. Include examples of [...], phone numbers, and German text labeled as non-formulas. https://learn.microsoft.com/en-us/azure/ai-services/document-intelligence/concept-custom
3. Lastly, about language-specific filtering, use Azure’s language detection API to exclude non-mathematical languages (e.g., German):

      from azure.ai.textanalytics import TextAnalyticsClient
      def is_german(text):
          client = TextAnalyticsClient(endpoint, credential)
          result = client.detect_language([text])
          return result[0].primary_language.iso6391_name == "de"
      
   

   https://learn.microsoft.com/en-us/azure/ai-services/language-service/language-detection/overview

Success