Skip to main content
Honeydew support for Google BigQuery is currently in Beta.Please contact [email protected] to get access to Google BigQuery integration.

​
Google BigQuery Integration Setup

Honeydew requires access to Google BigQuery to operate. You have two options to set up Google BigQuery access - either using a central org-level connection or map your individual Google BigQuery user credentials to Honeydew. If you would like to use a central org-level connection, it is advised to create a new dedicated service account for Honeydew integration. The following Google BigQuery connection parameters are required for Honeydew setup:
  1. Project ID
  2. Dataset (for storing dynamic datasets)

​
Authentication Methods

Honeydew supports the following authentication methods for Google BigQuery:

​
Service Account Key authentication

This is the recommended method for org-level service accounts. For this method, you will need to provide a Service Account JSON Key File.
1

Create a service account

If you do not already have a service account, create one in your Google Cloud project:
  1. Go to the Google Cloud Console
  2. Navigate to IAM & Admin > Service Accounts
  3. Click Create Service Account
  4. Enter a name for the service account (e.g. “Honeydew Integration”)
  5. Optionally add a description
  6. Click Create and Continue
2

Grant required permissions

Grant the service account the necessary IAM roles to access Google BigQuery:
  1. In the service account creation flow, under Grant this service account access to project
  2. Add the role BigQuery Job User (allows running queries)
  3. Click Continue
  4. Click Done
Additional dataset-level permissions will be configured in a later step
3

Generate and download the JSON key

To generate credentials for the service account:
  1. In the service accounts list, click the service account you just created
  2. Go to the Keys tab
  3. Click Add Key > Create new key
  4. Select JSON as the key type
  5. Click Create
  6. The JSON key file will be downloaded automatically
  7. Store this file securely
The JSON key file contains sensitive credentials. Store it in a secure location.
4

Configure the Google BigQuery connection in Honeydew

In Honeydew App settings page, configure the Google BigQuery connection by uploading the Service Account JSON Key File from the previous step, along with the Project ID.

​
OAuth User Authentication

This method allows individual users to authenticate with their Google credentials. Each user will need to authorize Honeydew to access their Google BigQuery resources using OAuth.
OAuth user authentication requires additional configuration. Contact [email protected] to enable this authentication method.

​
Connection Parameters

​
Project ID

The Project ID is the unique identifier for your Google Cloud project. You can find it in the Google Cloud Console:
  1. Go to the Google Cloud Console
  2. Select your project from the project drop-down at the top
  3. The Project ID is displayed in the project info panel
The Project ID is different from the project name. Make sure to use the Project ID, not the project name.

​
Dataset Configuration

You will need to specify:
  1. Dataset - the dataset where Honeydew will deploy dynamic datasets as views or tables
  2. Dev Dataset - the dataset for dev branch deployments (optional, defaults to main dataset)
Google BigQuery uses a hierarchical namespace: project.dataset.table. Ensure your service account has appropriate permissions to access these resources.

​
Allowing Honeydew Client IP Addresses

If you have IP-based access restrictions in Google BigQuery using VPC Service Controls, add the IP addresses displayed in the Google BigQuery connection screen in Honeydew App settings page to the access level allowlist.
For the Honeydew Cloud deployment, the following IP addresses are used:
  • 34.86.209.90
  • 34.145.147.92
If you are using a private Honeydew deployment, the IP addresses will be different. You can find them in the Google BigQuery connection screen in Honeydew App settings page.
To configure VPC Service Controls:
  1. Create an access level with the Honeydew IP addresses
  2. Add the access level to your service perimeter protecting Google BigQuery
  3. Test with a dry-run perimeter first to avoid blocking legitimate traffic

​
Permissions

Honeydew does not extract or store your data. It only reads schema metadata and executes SQL queries inside your Google BigQuery environment. You can find more security-related information here.

​
Required Permissions

If using a service account, the following permissions are required:

​
Project-Level Permissions

BigQuery Job User role (roles/bigquery.jobUser) - Required to run queries and jobs Grant this role at the project level: 
gcloud projects add-iam-policy-binding PROJECT_ID \
  --member="serviceAccount:SERVICE_ACCOUNT_EMAIL" \
  --role="roles/bigquery.jobUser"

​
Dataset-Level Permissions

For datasets used in the semantic layer:
  1. BigQuery Data Viewer role (roles/bigquery.dataViewer) - Read access to tables and views
  2. BigQuery Metadata Viewer role (roles/bigquery.metadataViewer) - Read dataset and table metadata
For the dataset where dynamic datasets will be deployed: BigQuery Data Editor role (roles/bigquery.dataEditor) - Create and manage tables/views Grant dataset-level permissions: 
# For datasets used in the semantic layer
bq add-iam-policy-binding \
  --member="serviceAccount:SERVICE_ACCOUNT_EMAIL" \
  --role="roles/bigquery.dataViewer" \
  PROJECT_ID:DATASET_NAME

# For the dataset where dynamic datasets will be deployed
bq add-iam-policy-binding \
  --member="serviceAccount:SERVICE_ACCOUNT_EMAIL" \
  --role="roles/bigquery.dataEditor" \
  PROJECT_ID:DATASET_NAME
For production deployments, consider creating a custom role with only the specific permissions needed rather than using predefined roles.

​
Minimal Permission Set

For a minimal permission set, the service account needs:
  • bigquery.jobs.create (at project level)
  • bigquery.tables.get, bigquery.tables.list, bigquery.tables.getData (on source datasets)
  • bigquery.tables.create, bigquery.tables.update, bigquery.tables.delete (on deployment dataset)
  • bigquery.datasets.get, bigquery.readsessions.create (on all datasets)

​
Tracking Honeydew Queries in Google BigQuery

You can track and monitor queries executed by Honeydew in Google BigQuery using query labels and the INFORMATION_SCHEMA views.

​
Query Label Format

All Honeydew queries include labels with the following format: 
{
  "application": "honeydew",
  "workspace": "some_workspace",
  "branch": "branch_name",
  "user": "username_example_com",
  "client": "honeydew_server"
}
Google BigQuery labels only support lowercase letters, numbers, hyphens, and underscores. Special characters like @ and . in email addresses are replaced with underscores.
The query labels contain:
  • application: Always set to “honeydew”
  • workspace: The Honeydew workspace name
  • branch: The Honeydew workspace branch being used (e.g., “dev”, “prod”)
  • user: The Honeydew user identifier (email address with special characters replaced)
  • client: The client name, usually “honeydew_server” for server-side operations

​
Tracking Methods

You can track Honeydew queries using any of the following approaches:

​
1. By Service Account

If you’re using a dedicated service account for Honeydew integration, you can filter queries by the service account email in the INFORMATION_SCHEMA.JOBS view. The most comprehensive method is to filter by Honeydew query labels. This approach allows you to track queries by workspace, branch, or user. Example query to track Honeydew queries: 
SELECT
  job_id,
  creation_time,
  user_email,
  total_bytes_processed,
  total_slot_ms,
  query
FROM
  `PROJECT_ID.region-REGION.INFORMATION_SCHEMA.JOBS`
WHERE
  ARRAY_LENGTH(labels) > 0
  AND EXISTS(
    SELECT 1
    FROM UNNEST(labels) AS label
    WHERE label.key = 'application' AND label.value = 'honeydew'
  )
ORDER BY
  creation_time DESC;

​
3. Filter by Specific Workspace or Branch

To track queries for a specific workspace: 
SELECT
  job_id,
  creation_time,
  user_email,
  query
FROM
  `PROJECT_ID.region-REGION.INFORMATION_SCHEMA.JOBS`
WHERE
  EXISTS(
    SELECT 1
    FROM UNNEST(labels) AS label
    WHERE label.key = 'workspace' AND label.value = 'your_workspace_name'
  )
ORDER BY
  creation_time DESC;
Replace PROJECT_ID and REGION in the queries above with your actual project ID and Google BigQuery region (e.g., us, eu).