how can i setup iceberg in databric

Question

how can i setup iceberg in databric

Kirk Anderson 0

I have a simple pay-as-you-go for learning purposes. I am trying to simulate Adobe Clickstream in Azure Databricks and furthermore trying to create an Iceberg table under Unity Catalog. Tried to utilize assisted AI to accomplish goal to no avail.

I have a feeling Iceberg not available in my current workspace configuration. How can i do this on the cheap? - have had no success. thank you

Chandra Boorla 12,085 Reputation points Microsoft External Staff

2025-04-22T17:41:35.7866667+00:00
@Kirk Anderson

Regarding on the query, you are correct in suspecting that Iceberg may not be available in your current setup - let me help clarify a few key points so you can get up and running as efficiently (and cost-effectively) as possible:

Iceberg on Unity Catalog – Requirements:

Unity Catalog must be enabled in your Databricks workspace, this is only available on Premium or Enterprise pricing tiers. Iceberg is not supported on the Standard tier.

Databricks Runtime 13.1+ is required for Iceberg, though version 14.0+ is recommended for improved stability.

If you are writing external Iceberg tables to ADLS Gen2, you'll need to:

Set up and configure External Locations

Enable external data access on the Unity Catalog metastore

Grant permissions like USE, CREATE TABLE, and EXTERNAL LOCATION

For Learning & Simulation:

If you are on a Standard tier (pay-as-you-go) workspace, you might not currently have Unity Catalog - and therefore won't be able to use Iceberg. In that case, you have two options:

Upgrade your workspace to the Premium tier so you can enable Unity Catalog and Iceberg.

Or, if you're just testing out data lake formats, consider using Delta Lake, which is fully supported even on Standard and Community Editions.

Example Iceberg Table (once setup is ready):

CREATE TABLE main.default.clickstream_events ( user_id STRING, timestamp TIMESTAMP, event_type STRING ) USING ICEBERG;

For additional information, please refer: Read Databricks tables from Iceberg clients

I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.

Thank you.
Chandra Boorla 12,085 Reputation points Microsoft External Staff

2025-04-23T17:38:56.7966667+00:00

@Kirk Anderson

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Chandra Boorla 12,085 Reputation points Microsoft External Staff

2025-04-24T16:11:38.7933333+00:00

@Kirk Anderson

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

1 answer

Your answer

Chandra Boorla 12,085 Reputation points Microsoft External Staff

2025-04-22T17:41:35.7866667+00:00

@Kirk Anderson

Regarding on the query, you are correct in suspecting that Iceberg may not be available in your current setup - let me help clarify a few key points so you can get up and running as efficiently (and cost-effectively) as possible:

Iceberg on Unity Catalog – Requirements:

Unity Catalog must be enabled in your Databricks workspace, this is only available on Premium or Enterprise pricing tiers. Iceberg is not supported on the Standard tier.

Databricks Runtime 13.1+ is required for Iceberg, though version 14.0+ is recommended for improved stability.

If you are writing external Iceberg tables to ADLS Gen2, you'll need to:

Set up and configure External Locations

Enable external data access on the Unity Catalog metastore

Grant permissions like USE, CREATE TABLE, and EXTERNAL LOCATION

For Learning & Simulation:

If you are on a Standard tier (pay-as-you-go) workspace, you might not currently have Unity Catalog - and therefore won't be able to use Iceberg. In that case, you have two options:

Upgrade your workspace to the Premium tier so you can enable Unity Catalog and Iceberg.

Or, if you're just testing out data lake formats, consider using Delta Lake, which is fully supported even on Standard and Community Editions.

Example Iceberg Table (once setup is ready):

CREATE TABLE main.default.clickstream_events ( user_id STRING, timestamp TIMESTAMP, event_type STRING ) USING ICEBERG;

For additional information, please refer: Read Databricks tables from Iceberg clients

I hope this information helps. Please do let us know if you have any further queries.

Kindly consider upvoting the comment if the information provided is helpful. This can assist other community members in resolving similar issues.

Thank you.
Chandra Boorla 12,085 Reputation points Microsoft External Staff

2025-04-23T17:38:56.7966667+00:00

@Kirk Anderson

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Chandra Boorla 12,085 Reputation points Microsoft External Staff

2025-04-24T16:11:38.7933333+00:00

@Kirk Anderson

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Answer 1

Hi there Kirk Anderson

Thanks for using QandA platform

guess you are right, Iceberg with Unity Catalog needs a Unity-enabled workspace and DBR 13.1+. On a pay-as-you-go plan, check that your metastore is set up and attached.

Try this to create an Iceberg table

CREATE TABLE catalog.schema.table_name (
  user_id STRING,
  event_time TIMESTAMP,
  page_url STRING
) USING ICEBERG;

Use a single-node cluster with Unity Catalog enabled to keep costs low. If Unity Catalog isn’t showing, your workspace might not support it yet.

If this helps kindly accept the answer thanks much.

Share via

how can i setup iceberg in databric

1 answer

Your answer