Deploy models for batch inference and prediction

This article describes what Databricks recommends for batch inference.

For real-time model serving on Azure Databricks, see Deploy models using Mosaic AI Model Serving.

AI Functions for batch inference

AI Functions are built-in functions that you can use to apply AI on your data that is stored on Databricks. You can run batch inference using task-specific AI functions or the general purpose function, ai_query.

The following is an example of batch inference using the task-specific AI function, ai_translate. If you want perform batch inference on an entire table, you can remove the limit 500 from your query.

SELECT
writer_summary,
  ai_translate(writer_summary, "cn") as cn_translation
from user.batch.news_summaries
limit 500
;

Alternatively, you can use the general purpose function, ai_query to perform batch inference.

• See which model types and the associated models that ai_query supports.
• See Perform batch LLM inference using AI Functions.

Batch inference using a Spark DataFrame

See Perform batch inference using a Spark DataFrame for a step-by-step guide through the model inference workflow using Spark.

For deep learning model inference examples see the following articles:

• Model inference using TensorFlow and TensorRT
• Model inference using PyTorch