Synthetic test environments in Unity Catalog let Informatica migration teams validate against realistic data without touching production.
Tagging strategy for Databricks
A mature data estate has an established tagging strategy for all cloud resources. We are going to focus more on how to establish a mature tagging strategy within the Databricks platform. A reoccurring question we help tackle is “how can we see” all tables or jobs for a given domain/topic/etc. As with your cloud resources, you can configure tag(s) for resources and assets within Databricks. Applying tags will help track processes and costs, and can be leveraged with a Data Catalog. We will step through the different areas you can configure the tags and how useful the tags can be once adoption has been established.
Table of Contents
Add a header to begin generating the table of contents
Table of Contents
Add a header to begin generating the table of contents
What are tags?
A tag is meta-data added to a given resource to help provide purpose, identification, grouping, tracking.
The hardest decision you will need to make is what buckets of information do you want to track and the values to be utilized. Yes, “buckets” is plural, for the more ways to categorize the information, the more ways you can drill into what is going on within your Databricks platform. A tag is configured as a key/value pair. Your key is the “bucket” or category to be used across a common set of resources. The value would be the individual results for the selected key. The great thing about tagging is you can add/remove over time as the strategy changes. The only exception is you cannot modify history.
Tags are key/value pairs.
- Key = the category or “bucket”
- Value = the identifier within that category
environment(e.g.,dev)-
department(e.g.,finance) -
domain(e.g.,financial_reporting)
- Use consistent casing (e.g., lowercase, underscores)
- Keep tags short but descriptive
- Avoid unclear abbreviations
Applications
Where can tags be applied?
Within the Databricks platform, we can apply tags on the following resources:
- All purpose clusters
- Job clusters
- Compute policies
- Cluster pools
- SQL Warehouses
- Jobs
- DLT Pipelines (via Cluster Policy for the compute)
- Databricks Apps (via Cluster Policy for the compute)
Limitations
There are certain scenarios when tags cannot be applied:
- Job API for one-time jobs
- Azure Data Factory Notebook Execution widget
Inherited tagging behavior
A few resources inherit the tags from the above tagging configuration on the resources mentioned:
- DLT Pipelines inherit tags from the cluster policy
- Databricks Apps inherit tags from the compute resource
- Workflows (Jobs) inherit tags from compute or cluster policy
Note: You can also directly assign tags to workflows.
Tagging examples
Here are a few examples of where tags can be added to the resource.
Tagging in Unity Catalog
With Unity Catalog, you can also tag the assets:
- Catalog
- Schema
- Table
- Column
- Volumes
- Models
Implementation
Four phases to implement a tagging strategy
Implementation of a tagging strategy can be bundled into a 4 phase approach:
- Define the categories or buckets and the anticipated values
- Document standards for tagging resources
- Identify priority resources needing tagging
- Start with new resources
- Tag high-priority existing resources
- Implement validation checks
- Create an automated tagging process for compute (Cluster Policies)
- Implement monitoring of untagged resources
- Create an AI/BI Dashboard based off system table queries
- Review tag usage patterns
- Adjust tag categories or buckets as needed
- Expand the tagging of existing resources
Tag utilization
Now we have tags associated with our Databricks resources and assets, what can we do to utilize them? There are system tables within Unity Catalog which help you produce queries to build a dashboard for users to quickly find assets and resources from the associated tags.
SELECT table_catalog, table_schema, table_name, department, environment
FROM
(
SELECT
ti.table_catalog,
ti.table_schema,
ti.table_name,
tt.tag_name,
tt.tag_value
FROM `system`.information_schema.tables ti
INNER JOIN `system`.information_schema.table_tags tt on ti.table_catalog = tt.catalog_name and ti.table_schema = tt.schema_name and ti.table_name = tt.table_name
) tbl
PIVOT
(
min(tag_value)
for tag_name in ('department', 'environment')
)
;
WITH job (
SELECT
workspace_id,
job_id,
name,
tags,
change_time,
rank() over (partition by workspace_id,job_id order by change_time desc) as rnk
FROM
system.lakeflow.jobs
)
SELECT
workspace_id, job_id, name, environment, department
FROM
(
SELECT
*
FROM
job LATERAL VIEW explode(tags) as k,
v
WHERE
rnk = 1
) tbl PIVOT (
min(v) FOR k IN ('environment', 'department')
);
Taking tagging further with the Lakehouse Optimizer
While Databricks provides foundational tagging support, the Lakehouse Optimizer (LHO) delivers a more advanced and actionable tagging experience. LHO indexes all available tags across your Databricks environment, enriches them with real-time cost attribution, and enables detailed chargeback reporting. It also provides granular visibility into untagged or “orphan” resources, helping teams quickly identify and resolve gaps. With organizational mapping built in, teams can analyze usage and spend not just by workspace but by business unit, project, or department—even when workloads span multiple environments. This business-aligned view gives FinOps and engineering leaders the insights they need to drive accountability, optimize resource allocation, and reduce waste at scale.
Conclusion
In conclusion, a well-designed tagging strategy is fundamental for managing a modern data estate within Databricks. Consistently applying tags across compute resources, workflows, and the data assets, organizations gain visibility, control, and insight into the environment. When implementing tagging, it does require initial planning and discipline, the benefits in terms of resource management, cost allocation, and governance make it an essential practice.
Even a simple tagging strategy when applied consistently will provide significant value over no tagging at all. Start small, build, and expand the tagging strategy as the needs evolve.
Share with your network
You may also enjoy
Palantir holds the decisions. Databricks runs the compute. Which work belongs where is now a question you can measure instead of argue.
