How OKF gives domain experts a governed way to contribute definitions to Databricks Genie
A Databricks Genie Agent can make governed data easier to use, but trustworthy answers still depend on curation. The person who builds the Genie Agent needs to understand the data, metric logic, business terminology, SQL, example queries, and instructions that shape the agent’s behavior. The interesting problem begins when the person who knows the business meaning is not the person who can configure Genie.
The person asking questions should not need to know SQL, understand table schemas, or guess which column to filter. Databricks describes this relationship clearly: the quality of a Genie Agent is determined by the curator, not the consumer. When answers are wrong, the normal remedy is better curation, not better questions.
That model is sound. In tightly controlled organizations, however, it leaves a practical gap: business experts often know what a definition should mean, while curators know how to implement it. This article is about the bridge between those two kinds of knowledge.
The vocabulary in one minute
Before getting into the workflow, here are the terms used in this article:
The short version is: OKF records the meaning, the curator implements it, Unity Catalog governs it, and Genie delivers it.
Terms used in this article
Genie Agent
A Databricks conversational analytics experience configured for a particular business domain. It uses governed data, instructions, examples, metrics, rules, and verified answers to produce responses.
Space curator
The person who builds and maintains the Genie Agent. The curator understands the data and business meaning and has the SQL skill to encode that knowledge.
Space consumer
The person who asks questions. They should be able to use the space without knowing SQL or the underlying table design.
OKF
Google Open Knowledge Format. In this article, it means a reviewable Markdown and YAML contract for recording business definitions, examples, policy context, ownership, provenance, and lifecycle state.
Unity Catalog (UC)
Databricks’ governance layer for data and AI assets, including permissions, lineage, ownership, discovery, and controlled access.
Metric View
A governed Unity Catalog object that defines reusable measures, dimensions, joins, filters, and metric semantics for production use.
Semantic definition
The business meaning behind a term or metric: what it includes, excludes, measures, and how it should be used.
The curator-consumer gap
The person who understands a business definition best is not always the person who can edit Genie. A controller may know exactly what “adjusted margin” means. A planner may know when inventory is eligible for replenishment. A product manager may have a definition they want the organization to adopt and evangelize. The business has the answer in someone’s head, a spreadsheet, or a well-loved slide deck. Genie, understandably, cannot treat any of those as a governed runtime contract yet.
That subject-matter expert may not have permission to edit Genie instructions, attach data sources, write SQL expressions, or create a Unity Catalog Metric View. Meanwhile, the authorized curator or data engineering team may have the technical skills to implement the definition without knowing all of its business meaning, edge cases, and policy implications.
This is not a failure of either role. It is a boundary between semantic authorship and runtime curation. One person knows the meaning; another person owns the machinery. Good governance needs both.
OKF as the contribution layer
Google Open Knowledge Format, used here as a governed Markdown and YAML contract pattern, fills that gap. A domain expert can document a proposed definition in a reviewable artifact without directly changing the production Genie Agent. It is a semantic handoff, not a back door into production.
An OKF concept can capture:
- the business definition and accepted terminology;
- examples, exclusions, and edge cases;
- the intended source or Metric View;
- calculation notes and policy context;
- ownership, provenance, version, and effective dates; and
- review, verification, activation, and deprecation state.
The curator or data engineering team then validates and formalizes the approved concept in the runtime surfaces: Genie instructions, SQL expressions, sample queries, governed source mappings, or a Unity Catalog Metric View.
The flow preserves Databricks’ responsibility model. The domain expert contributes meaning. The curator owns implementation, testing, permissions, and production configuration. The Genie consumer receives the governed result. Everyone gets to keep their proper job; nobody has to become an accidental SQL author overnight.
What remains authoritative
OKF is not a replacement for Databricks governance. It is a contribution and review layer upstream of the runtime authority.
- OKF captures proposed meaning, policy context, provenance, and review state.
- Unity Catalog Glossary governs shared business terms and definitions.
- Unity Catalog Metric Views hold canonical production metric logic.
- Genie Agent configuration provides the curated instructions, trusted assets, examples, rules, and verified answers.
- Unity Catalog permissions and policies remain the enforcement boundary.
An approved OKF concept still has to pass the organization’s review, source validation, trust, freshness, and activation gates before it can influence production answers.
A concrete example: adjusted margin
Imagine a finance team wants to introduce an adjusted-margin definition. The finance consumer knows that the metric excludes one-time restructuring charges, uses a specific fiscal period, and must not be used for statutory reporting. The curator knows how to map the definition to governed tables and how to express the calculation in a Metric View. The two roles are solving different halves of the same problem.
The OKF contract makes the finance team’s definition explicit and reviewable. The curator validates the source columns, implements the approved SQL/YAML logic, adds the relevant Genie instructions and examples, and tests the result. Later, a new definition can supersede the old one with its own version, effective date, and approval trail. The definition can evolve without relying on memory or hunting through old chat threads.
This is more reliable than leaving the definition in email, chat, or an informal prompt because the semantic intent becomes a durable artifact that can be reviewed alongside its implementation.
The narrow claim
OKF does not make consumers responsible for fixing Genie answers, and it does not remove the need for knowledgeable curators. It fills the translation gap between the person who knows what a definition should mean and the person authorized to implement it.
Once approved and formalized, the governed Databricks runtime remains the source of truth. The benefit is that domain expertise is no longer trapped behind editing permissions or lost in informal conversations. It becomes a versioned, reviewable input to better Genie curation.
Read the full white paper
The complete technical treatment, 56 pages, covers the contract registry, lifecycle and trust states, source-of-truth hierarchy, conflict testing, prompt-injection boundaries, and appendices A to H on metric-view syntax and lifecycle controls.
References
Author note and disclaimer
The views and opinions expressed in this document are those of the author and are based on information available at the time of writing. Technology platforms, product capabilities, industry best practices, and architectural patterns evolve rapidly and may change over time. Any product references, examples, or implementation approaches are provided for illustrative and educational purposes only and should not be considered official vendor guidance, endorsements, commitments, or future roadmaps. Readers should evaluate all recommendations within the context of their organization’s technical, operational, governance, security, and regulatory requirements.
Saket Suman is a Solutions Architect specializing in Data, AI, and GenAI, with 23+ years of experience across data architecture, machine learning, analytics, cloud platforms, and secure enterprise AI solutions, including leadership roles at Microsoft and Blueprint LLC. He writes extensively about Databricks and AI, sharing insights on topics such as Unity Catalog, data governance, GenAI, machine learning, and modern data platform architecture.
