Chat completion endpoint with vision model returning invalid image data

Hi srikanth penmetsa

Could you share your input image in private chat for replication at our side.

I am able to get results and below sample code.

Results

ChatCompletion(id='chatcmpl-BSlIeq3cYn8QGtmTmMs0wXr3raZrP', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content="The image is a map that depicts a route between two points labeled 'S' and 'D'. The route is marked by lines of different colors indicating different segments of the journey:.....
.....

for below code and image

import os
from openai import AzureOpenAI

api_base = '<baseurlfromopenai>' # your endpoint should look like the following https://YOUR_RESOURCE_NAME.openai.azure.com/
api_key="<apikey>"
deployment_name = 'gpt-4o'
api_version = '2025-01-01-preview' # this might change in the future

client = AzureOpenAI(
    api_key=api_key,  
    api_version=api_version,
    base_url=f"{api_base}openai/deployments/{deployment_name}",
)

import base64
from mimetypes import guess_type

# Function to encode a local image into data URL 
def local_image_to_data_url(image_path):
    # Guess the MIME type of the image based on the file extension
    mime_type, _ = guess_type(image_path)
    if mime_type is None:
        mime_type = 'application/octet-stream'  # Default MIME type if none is found

    # Read and encode the image file
    with open(image_path, "rb") as image_file:
        base64_encoded_data = base64.b64encode(image_file.read()).decode('utf-8')

    # Construct the data URL
    return f"data:{mime_type};base64,{base64_encoded_data}"

# Example usage
image_path = 'C:/Users/v-manmohanty/Pictures/Screenshots/mapsample.png'
data_url = local_image_to_data_url(image_path)
#print("Data URL:", data_url)

response = client.chat.completions.create(
    model=deployment_name,
    messages=[
        { "role": "system", "content": "You are a helpful assistant." },
        { "role": "user", "content": [  
            { 
                "type": "text", 
                "text": "Describe this picture:" 
            },
            { 
                "type": "image_url",
                "image_url": {
                    "url": data_url
                }
            }
        ] } 
    ],
    max_tokens=2000 
)

content
 
print

Input image

Reference used

https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/chatgpt?tabs=python&pivots=programming-language-chat-completions

Thank you.