Model Deployment Error – “Must specify a chain type in config” while serving LLM-based Chatbot via Unity Catalog

@Muhammad Salman Khan

Please check below steps and confirm us

Ensure that you have explicitly defined the chain type in your model configuration. For example, if you are using LangChain, you might need to specify chain_type="stuff" or another appropriate chain type when building your model

Verify that the model is fully registered and ready for serving. Sometimes, the model might appear as "Not Ready" due to incomplete registration. Double-check the registration status in Unity Catalog

Ensure that the MLflow version used during model logging is compatible with the version used during model serving. In some cases, updating the MLflow version or adjusting the request format can resolve compatibility issues

Double-check other configuration settings such as scale_to_zero_enabled, workload_type, and workload_size to ensure they match your intended setup

If these steps don't resolve the issue, please share more details or logs with screenshots if possible

@Muhammad Salman Khan We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

@phemanth

Thank you so much for your response. I have verified all the steps that you mentioned. Attaching the snippets for your reference.

Model is completely registered and ready to use,

Chain type is explicitly mentioned.

I am using the following code to register the model:

import mlflow
from mlflow.models import infer_signature

# Set the registry URI (adjust to your Unity Catalog setup if needed)
mlflow.set_registry_uri("databricks-uc")

# Generalize model name for broader reuse
model_name_last = 'baseline_rag_2304_1440'
model_name = f"dev.ai.{model_name_last}"

# Example input/output for signature
question = "Question Text"
answer = chain.invoke(question)

with mlflow.start_run(run_name="your_project_rag_run_1") as run:
    signature = infer_signature(question, answer)

    model_info = mlflow.langchain.log_model(
        chain,
        loader_fn=get_retriever,           # Will be used during model loading
        artifact_path="chain",
        registered_model_name=model_name,
        input_example=question,
        signature=signature,
        extra_pip_requirements=[
        "langchain==0.3.23",
        "openai==1.75.0",
        "langchain-openai==0.3.14",
        "databricks-sdk==0.50.0",
        "databricks-vectorsearch==0.56",
        "langchain-community==0.3.21",
        "databricks-langchain==0.4.2"]
    )

The model gets registered fine, but when I try to serve it or even call it in the notebook from the unity catalog, I get the following error: