Azure OpenAI Vector Store - Attributes Retrieval

Eric S 0 Reputation points
2025-04-28T20:30:14.0466667+00:00

I'm having trouble to retrieve attributes from my vector store, it fails at the retrieve. The same code works with OpenAI. I'm using 2025_03_02-preview API version.

Code:
import os

import sys

import tempfile

import time

from openai import AzureOpenAI, APIError, NotFoundError

from dotenv import load_dotenv

load_dotenv()

API_KEY = os.getenv("AZURE_OPENAI_API_KEY")

ENDPOINT = os.getenv("AZURE_OPENAI_API_BASE")

API_VERSION = os.getenv("AZURE_OPENAI_API_VERSION", "2025-03-01-preview")

if not API_KEY or not ENDPOINT:

print("Missing AZURE_OPENAI_API_KEY or AZURE_OPENAI_API_BASE in environment.")

sys.exit(1)

client = AzureOpenAI(api_key=API_KEY, api_version=API_VERSION, azure_endpoint=ENDPOINT)

VECTOR_STORE_NAME = f"debug_vs_{int(time.time())}"

TEST_ATTRIBUTE = {"test_key": "test_value"}

SAMPLE_TEXT = "This is a test file for vector store attribute debugging."

try:

1. Create vector store with metadata

print(f"Creating vector store: {VECTOR_STORE_NAME}")

vs = client.vector_stores.create(name=VECTOR_STORE_NAME, metadata=TEST_ATTRIBUTE)

vs_id = vs.id

print(f"Created vector store with ID: {vs_id}")

print(f"Vector store metadata: {getattr(vs, 'metadata', None)}")

2. Create a temp file and upload

with tempfile.NamedTemporaryFile(mode='w+', encoding='utf-8', suffix=".txt", delete=False) as temp_f:

temp_f.write(SAMPLE_TEXT)

temp_file_path = temp_f.name

print(f"Uploading file: {temp_file_path}")

with open(temp_file_path, "rb") as file_to_upload:

file_obj = client.files.create(file=file_to_upload, purpose="assistants")

file_id = file_obj.id

print(f"Uploaded file with ID: {file_id}")

3. Add file to vector store with attribute

print(f"Adding file to vector store with attributes: {TEST_ATTRIBUTE}")

vs_file = client.vector_stores.files.create(

vector_store_id=vs_id,

file_id=file_id,

attributes=TEST_ATTRIBUTE

)

vs_file_id = vs_file.id

print(f"Added file to vector store. VSFile ID: {vs_file_id}")

Wait for file to be processed (status 'completed')

print("Waiting for file to be processed (status 'completed')...")

max_wait = 60 # seconds

wait_time = 0

poll_interval = 3

while wait_time < max_wait:

retrieved_vs_file = client.vector_stores.files.retrieve(vector_store_id=vs_id, file_id=file_id)

status = getattr(retrieved_vs_file, 'status', None)

print(f" Status: {status} (waited {wait_time}s)")

if status == 'completed':

break

time.sleep(poll_interval)

wait_time += poll_interval

else:

print("Timeout waiting for file to be processed. Exiting test.")

raise RuntimeError("File did not reach 'completed' status in time.")

4. Retrieve file from vector store and check attribute

print("Retrieving file from vector store...")

print(f"Full retrieved_vs_file object: {repr(retrieved_vs_file)}")

if hasattr(retrieved_vs_file, 'dict'):

print(f"retrieved_vs_file.dict: {retrieved_vs_file.dict}")

attrs = getattr(retrieved_vs_file, 'attributes', None)

print(f"Retrieved attributes: {attrs}")

if attrs and attrs.get("test_key") == "test_value":

print("PASS: Attribute was correctly stored and retrieved.")

else:

print("FAIL: Attribute was not correctly stored or retrieved.")

except Exception as e:

print(f"Error during test: {e}")

finally:

Cleanup: delete file and vector store

try:

print("Cleaning up: deleting file and vector store...")

if 'file_id' in locals():

client.files.delete(file_id=file_id)

if 'vs_id' in locals():

client.vector_stores.delete(vector_store_id=vs_id)

if 'temp_file_path' in locals() and os.path.exists(temp_file_path):

os.remove(temp_file_path)

except Exception as cleanup_e:

print(f"Cleanup error: {cleanup_e}")

Azure AI Search
Azure AI Search
An Azure search service with built-in artificial intelligence capabilities that enrich information to help identify and explore relevant content at scale.
1,283 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Prashanth Veeragoni 4,030 Reputation points Microsoft External Staff
    2025-04-28T21:55:38.1666667+00:00

    Hi Eric S,

    Thanks for explaining everything so clearly.

    You're very close — but the issue you're facing is actually due to a limitation of the current Azure OpenAI 2025-03-02-preview API version:

    Azure OpenAI Vector Stores currently do not fully support attributes retrieval yet, even though OpenAI API does!

    Why is it happening?

    ·       In OpenAI's own API (api.openai.com), when you add attributes while uploading a file to a Vector Store, you can later retrieve them via the vector_stores.files.retrieve call.

    ·       However, in Azure OpenAI, attributes are currently accepted during creation, but NOT returned on retrieval yet.

    ·       This is a known gap as of the 2025-03-02-preview API version.

    That's why:

    ·       vs_file creation succeeds

    ·       but retrieved_vs_file does NOT contain the attributes you saved.

    It isn't your code's mistake — it's Azure's Vector Store backend not returning attributes yet.

    Confirmed by documentation and tests:

    ·       If you check the official Azure OpenAI Vector Store documentation you’ll notice they don't mention support for attributes in retrieve response yet.

    ·       OpenAI Python SDK works because OpenAI (non-Azure) backend supports this fully.

    How you can handle it temporarily:

    Manually store metadata elsewhere (workaround):

    Since Azure is not returning attributes yet, you can store the file_id and its attributes separately.

    Example:

    ·       When you add a file to the Vector Store, also save file_id + attributes mapping somewhere (like in a small Azure Table Storage, CosmosDB, or even a local JSON file).

    ·       Later, when you retrieve the file by ID, you can manually look up the attributes.

    Quick sample idea:

    file_metadata_store = {}  # Simple dictionary
    # After adding the file:
    file_metadata_store[file_id] = TEST_ATTRIBUTE
    # Later when retrieving:
    retrieved_attributes = file_metadata_store.get(file_id)
    
    

    Wait for Azure API to upgrade:

    Azure will eventually add full support for attribute retrieval (most likely in a later stable API version).

    Hope this helps, do let me know if you have any further queries. 

    Thank you!

    1 person found this answer helpful.
    0 comments No comments

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.