dbt

Honeydew dbt integration allows to materialize Honeydew-managed datasets via dbt.

​

Usage

Given a dynamic dataset named my_dataset_name create a dbt model called my_dataset_name.sql.

Put the following in the model code:

-- Optional: Set up various materialization parameters
{{ config(materialized='table') }}

-- Set up any additional dependencies in dbt with
-- depends_on: {{ ref('upstream_parent_model') }}

-- Dynamic dataset materialization
{{ honeydew.get_dataset_sql('my_dataset_name') }}

This model is now based on semantics defined in Honeydew.

​

Honeydew dbt Package

Honeydew provides a dbt package to integrate Honeydew with dbt.

​

High-Level Flow

dbt connects to Honeydew via a Snowflake connection and uses the Honeydew Snowflake Native Application to communicate with Honeydew.

In each run of a dbt model based on Honeydew:

1. dbt connects to Snowflake with the current profile.

2. Within a dbt model, the Honeydew dbt macro initiates a Snowflake call to Honeydew API.

   The macro uses the workspace and branch based on the Honeydew variable values set up in the dbt project

3. The Honeydew API client in Snowflake communicates with Honeydew to receive the correct compiled SQL for that branch.

   The Snowflake user and role set up in the dbt profile will be used to access Honeydew.

   Make sure that the role you use for dbt has access to the Honeydew native application.
   A role can be added with

   GRANT APPLICATION ROLE SEMANTIC_LAYER_ENTERPRISE_EDITION.HONEYDEW_APP_PUBLIC TO ROLE DBT_USER_ROLE;
   

   See installation guide for more details.

4. The compiled result of the macro call is the SQL for a Honeydew-based dbt model.

   The compiled SQL for a model based on Honeydew is generated for each dbt run, keeping it up to date with the semantic model definitions.

​

Installation

Honeydew runs in dbt projects connected to Snowflake. To interact with Honeydew, dbt can call the Honeydew Native Application.

Install the Honeydew dbt package by adding to packages.yml of your dbt project:

packages:
  - git: "https://github.com/honeydew-ai/honeydew-dbt.git"
    revision: "latest"

Run dbt deps to apply.

Set the following variables in the dbt project configuration (dbt_project.yml):

An example dbt_project.yml configuration:

vars:
  "honeydew:workspace": "tpch_demo"
  "honeydew:branch": "prod"
  "honeydew:snowflake_native_app_name": "SEMANTIC_LAYER_ENTERPRISE_EDITION"

​

Testing the integration

Communication details between dbt and Honeydew can be tested with the following call:

dbt -d run-operation honeydew.get_dataset_sql --args "{'dataset_name':'my_dataset_name'}" --profile snowflake

That call will provide the details of the operation done by Honeydew. Replace the snowflake profile with your dbt profile and my_dataset_name with the dataset name to test.

​

Extending the integration

Any Honeydew API operation can be added in the following way:

{{ honeydew.snowflake_native_app_call(<call_name>, var('honeydew:workspace'), var('honeydew:branch'), <call_parameters>, ...) }}`

​

Metadata Sync

Honeydew dbt metadata sync will support syncing with dbt the following metadata:

• Source dbt model references (when source dbt model is set for entities)
• Column descriptions
• Labels (as tags)
• General purpose metadata (if set)

​

Honeydew and different dbt offerings

​

dbt Core

Honeydew provides a dbt package that integrates with dbt. Using that package, Honeydew can manage dbt models in dbt Core.

Running dbt is based on an external orchestrator such as Airflow.

​

dbt Cloud

Works with Honeydew in the same way as dbt Core.

There is no need for an external orchestrator. Additional integration is available to link dbt documentation built on a dbt Cloud job and the Honeydew UI .

​

dbt Semantic Layer

There is no necessity to use the dbt Semantic Layer when using Honeydew. Honeydew metrics can connect to any BI tool directly.

Both can be used in parallel.

There are a number of core differences, such as:

1. Composable complex metrics: Honeydew enables a high degree of metric composition, including applying filters, partial aggregations, fan-out joins, multi-hop joins, multiple aggregation steps, time-based logic, and more.
2. Context-aware metrics: Honeydew metrics can adjust the user grouping, for example for dynamic ratios or dynamic nested aggregation.
3. Reuse metrics in data marts: Honeydew metrics can be directly used to build columns in other tables.
4. BI connectivity and Metadata Sync: Honeydew has standard SQL and metadata integrations that support a live connection with most major BI tools.
5. Security Model: Honeydew is designed around a user-based security model supporting row-level security and SSO.
6. Performance: Honeydew has a number of semantic-aware performance optimization such as aggregate awareness.

​

FAQ