Entity Caching
Entities may have calculated attributes that are expensive to compute - with aggregations, JOINs or large table scans. By default, every access to a calculated attribute recomputes it. However, for most calculated attributes recomputation is not necessary. Entity cache enables materializing entities, in order to avoid re-computation. The cache is updated upon triggers or according to a set schedule.By employing calculated attributes for multi-step calculations can build complex ELT data transformations directly in Honeydew.
In addition to Snowflake tables, entities can also be cached in supported BI tools that have memory capacity for cached data.See Power BI data caching for entity caching in Power BI.
Entity caches are singlular and are shared across domains and branches.
Configuration
Set up in Entity YAML schema cache delivery settings for Snowflake:Configuration when using dbt as orchestrator
Set up in Entity YAML schema cache delivery settings for dbt:Orchestration
Entity Cache refresh relies on external orchestration (with dbt or otherwise).
Set up orchestration in an external tool
Use the Snowflake Native Application API to get SQL for entity cache:Set up with dbt
To set up dbt as a cache orchestrator:- In dbt, create an entity cache model by using the Honeydew dbt cache macro
- In dbt, use the
config
macro to set up materialization settings such as clustering or dynamic tables - In Honeydew, set the entity’s dbt delivery settings to the chosen dbt model name
customers
model in dbt:
is_incremental()
dbt function may be used in combination with the Honeydew SQL macro for incremental caches.
However, make sure to check whether the computation itself had changed between runs, to avoid mixing different version
of logic in the same table.