dbt Projects - Lightdash

Lightdash supports dbt v1.4.0 and above. If you are using an older version of dbt, you will need to upgrade to sync your project to Lightdash.

Syncing your dbt project to Lightdash

There are five ways to sync changes from dbt into Lightdash. They differ in where the code comes from, where compilation happens, which profiles.yml / target governs the result, and whether they respect the project’s UI Connection Settings. For ready-to-use workflow templates, see the lightdash/cli-actions repo.

Sync mechanisms

1. lightdash deploy from a developer's laptop

Compiles the dbt project locally and pushes the resulting manifest.json to Lightdash.

Property	Value
dbt code source	Local working copy (uncommitted changes included)
Compile location	Laptop
`profiles.yml` source	Local (`~/.dbt/profiles.yml` by default, or `--profiles-dir`)
Target	Default in that `profiles.yml`, overridden by `--target`
UI Connection Settings respected?	No
Auth	`lightdash login` session or `LIGHTDASH_API_KEY` env var
Prerequisites	Lightdash CLI installed; dbt installed; warehouse creds reachable locally
Remote git connection required?	No

Use cases: initial project setup, debugging, one-off manual deploys.Trade-offs: bypasses PR review; relies on per-developer profiles.yml; high footgun risk if a developer’s prod target accidentally points at a non-prod warehouse.

2. lightdash deploy from CI

Same as #1, but runs inside a CI workflow (GitHub Actions, GitLab CI, Azure Pipelines, Bitbucket Pipelines, etc.) with credentials stored as CI secrets.

Property	Value
dbt code source	CI runner’s checkout of the repo
Compile location	CI runner
`profiles.yml` source	Provided by the workflow — either committed as an `env_var()` template, or written to disk from a CI secret at runtime
Target	Default in the CI `profiles.yml`, overridden by `--target`
UI Connection Settings respected?	No
Auth	`LIGHTDASH_API_KEY` from CI secrets
Prerequisites	Lightdash CLI + dbt installed in the workflow; warehouse creds available to CI; valid `profiles.yml` produced at runtime
Remote git connection required?	No
Workflow template	`deploy.yml` in `lightdash/cli-actions` (GitHub Actions example)

Use cases: automated production deploys gated by PR merge; PR preview workflows (lightdash start-preview + lightdash validate).Trade-offs: requires duplicating warehouse credentials into CI on top of what’s already in Lightdash; needs dbt available in the runner.

Never commit a profiles.yml with plaintext credentials. Either use an env_var() template committed to the repo, or write the file to disk at runtime from a CI secret.

3. "Refresh dbt" button in the Lightdash UI

Lightdash fetches the dbt code from the configured remote, compiles it server-side, and updates the project.

Property	Value
dbt code source	Lightdash pulls from the remote git source configured in Project Settings (GitHub, GitLab, Azure DevOps, Bitbucket, dbt Cloud)
Compile location	Lightdash servers
`profiles.yml` source	Lightdash synthesises one from Project Settings → Connection Settings
Target	UI Project Settings → Connection Settings → Target name
UI Connection Settings respected?	Yes
Auth	Logged-in user in the UI
Prerequisites	Project must have a remote git source connected
Remote git connection required?	Yes

Use cases: ad-hoc refresh after a merge; quick UI-driven sync without leaving the browser.Trade-offs: manual; relies on a human remembering to click it.

4. lightdash refresh from CI

Calls the same endpoint as the “Refresh dbt” button, but programmatically from CI (GitHub Actions, GitLab CI, Azure Pipelines, Bitbucket Pipelines, etc.). Lightdash does the work server-side using the project’s UI Connection Settings.

Property	Value
dbt code source	Lightdash pulls from the remote git source (configured in Project Settings)
Compile location	Lightdash servers
`profiles.yml` source	Synthesised by Lightdash from UI Connection Settings
Target	UI Project Settings → Connection Settings → Target name
UI Connection Settings respected?	Yes
Auth	`LIGHTDASH_API_KEY` from CI secrets
Prerequisites	Project must have a remote git source connected; Lightdash CLI installed in the workflow
Remote git connection required?	Yes
Workflow template	`refresh.yml` in `lightdash/cli-actions` (GitHub Actions example)

Use cases: automated post-merge sync to production when the project is already git-connected.Trade-offs: can’t push uncommitted changes (only what’s in the connected branch); won’t catch issues pre-merge — pair with a separate lightdash validate PR workflow if you want that.

5. Pinging the refresh API directly from CI

Same as #4, but bypasses the Lightdash CLI entirely — just a curl to the API. Works from any CI system (GitHub Actions, GitLab CI, Azure Pipelines, Bitbucket Pipelines, etc.).

Property	Value
dbt code source	Lightdash pulls from the remote git source (configured in Project Settings)
Compile location	Lightdash servers
`profiles.yml` source	Synthesised by Lightdash from UI Connection Settings
Target	UI Project Settings → Connection Settings → Target name
UI Connection Settings respected?	Yes
Auth	`LIGHTDASH_API_KEY` from CI secrets
Prerequisites	Project must have a remote git source connected
Remote git connection required?	Yes
Lightdash CLI required?	No
Endpoint	`POST /api/v1/projects/{projectUuid}/refresh`

Use cases: the simplest possible automated refresh — no dependencies to install. Useful for teams who don’t want a Node toolchain in CI.Trade-offs: same as #4. The refresh is async (returns a jobUuid); for stronger guarantees, poll GET /api/v1/jobs/{jobUuid} to confirm completion before exiting the workflow.

`lightdash deploy` vs `lightdash refresh`

The two CLI commands above (lightdash deploy and lightdash refresh) are often confused. Here’s the quick difference:

	`lightdash deploy`	`lightdash refresh`
What it does	Pushes your local dbt project to a Lightdash project. Runs `dbt compile` (or `dbt list`) locally to produce a fresh `manifest.json`, then sends the compiled models/explores to the Lightdash server.	Tells the Lightdash server to re-pull from the connected git remote and recompile, server-side. No local dbt run, no local files involved — it’s the API-driven equivalent of clicking Refresh dbt in the UI.
Where compilation happens	Locally (laptop or CI runner)	Lightdash servers
Requires a remote git connection?	No	Yes
Flags	Many (`--project`, `--select`, `--full-refresh`, `--create`, `--use-batched-deploy`, etc.)	Essentially none (just `--verbose`)
Use when	You’ve changed dbt models or Lightdash YAML metadata locally and want those changes to land in a Lightdash project.	Lightdash already points at a git repo and you just want it to pull the latest.

Rule of thumb:

Local dbt code you want to push → deploy
Lightdash already points at a git repo and you just want it to pull the latest → refresh

Decision matrix

	#1 `lightdash deploy` — local CLI	#2 `lightdash deploy` — CI	#3 “Refresh dbt” — UI button	#4 `lightdash refresh` — CI	#5 `POST /refresh` API — CI
Automated?	No	Yes	No	Yes	Yes
Pushes uncommitted local changes?	Yes	No	No	No	No
Needs dbt + warehouse creds in CI?	N/A	Yes	No	No	No
Needs Lightdash CLI installed?	Yes	Yes	No	Yes	No
Respects UI Target name?	No	No	Yes	Yes	Yes
Works without a remote git connection?	Yes	Yes	No	No	No
Catches breaking changes pre-merge?	With `--select` + `validate`	If used in a PR workflow	No	No	No
Best for	Initial setup, debugging	PR previews; CI deploys with no remote git connection	Ad-hoc UI refresh	Post-merge sync (recommended)	Post-merge sync, minimal-CI shops

Recommended setups

Most teams (Lightdash connected to GitHub, GitLab, Azure DevOps, Bitbucket, or dbt Cloud):

PR workflow → lightdash start-preview + lightdash validate (mechanism #2 with start-preview)
Post-merge to main → lightdash refresh (#4) or the refresh API (#5)
Manual UI refresh (#3) available for ad-hoc syncs

Teams without a remote git connection (project loaded via local CLI only):

Production sync → lightdash deploy in CI (#2)
Previews → lightdash start-preview in CI

Teams just getting started:

Local lightdash deploy (#1) for first-time setup, then graduate to one of the automated patterns above as soon as the project is stable.

Things to know

lightdash deploy does NOT use the UI “Target name” setting. It uses the local profiles.yml default, or whatever --target overrides it with. Only mechanisms #3, #4, and #5 respect the UI Target name.

lightdash refresh requires a remote git connection. It will fail with a ParameterError if the project’s dbtConnection.type is NONE.

Refresh is async. Both #4 and #5 return a jobUuid and the work continues server-side. If your CI needs to know the refresh succeeded, poll GET /api/v1/jobs/{jobUuid} before exiting the workflow.

Never commit profiles.yml with credentials. For mechanism #2, use either a secret-written-to-file approach or an env_var() template — but never check in plaintext credentials.

Mechanisms #4 and #5 are underused. Most existing CI examples lead with lightdash deploy, which is the heaviest option in CI (requires dbt + warehouse credentials duplicated into the runner). The refresh-based patterns are simpler and often a better fit when the project is git-connected.

How to trigger each sync

Click “Refresh dbt” in Lightdash

The button can be found on the Query from tables page.

If you’re using a git connection (like GitHub, GitLab, or Bitbucket), you’ll need to push + merge your changes to the branch that your Lightdash project is connected to before you press Refresh dbt.

If you’ve made any changes to the underlying data (for example, adding a new column in your model.sql file or changing the SQL logic of a dimension), then you need to run dbt run -m yourmodel before you click Refresh dbt in Lightdash.

Push code from the CLI

If you’re using the Lightdash CLI, you can use lightdash deploy to deploy your changes to Lightdash. Read more about how to use lightdash deploy.

We don’t recommend using lightdash deploy from your local environment as the primary way you update Lightdash since small mistakes can lead to production issues.

Set up continuous deployment

Read how to do that and check out our example workflow files.

dbt project settings

For more information about dbt connection types (Github, Gitlab, Bitbucket, etc.) and the fields required for each type, read the dbt project section in our connection guide. Below are details about the universal fields for all connected dbt projects.

dbt selector

This setting is found under Advanced configuration options

You can filter out models in your dbt project that you don’t want to see in Lightdash. This is useful if you have a large dbt project and you want to speed up the sync process. Unlike table selection, this selector is applied to the dbt models, so it will skip the entire compilation process for the models that you don’t want to see in Lightdash. To do this, you can add a dbt selector to your project settings. This is a JSON object that contains the models you want to include in Lightdash. For example, if you only want to include the my_model and all models with the lightdash tag in Lightdash, you can add the following to your dbt project settings:

my_model tag:lightdash

We support all dbt selectors. Read more about selectors in the dbt docs.

dbt targets

What is a dbt target?

A dbt target is an environment configuration in your dbt project that defines where and how dbt should run. In dbt Core, targets are defined in your profiles.yml file and typically include settings like:

Database connection details
Schema/dataset names
Warehouse specifications
Other environment-specific configurations

Most teams have multiple targets for different environments:

dev - for development work
prod - for production deployments
staging - for testing before production

Each target can have different connection settings which are stored in your profiles.yml file. The example profiles.yml file below shows different connection settings for the dev and prod targets. Note that on line 3 target: dev indicates that the dev target is the default target for this project.

# example profiles.yml file
jaffle_shop:
  target: dev
  outputs:
    dev:
      type: postgres
      host: localhost
      user: alice
      password: <password>
      port: 5432
      dbname: jaffle_shop
      schema: dbt_alice
      threads: 4

    prod:  # additional prod target
      type: postgres
      host: prod.db.example.com
      user: alice
      password: <prod_password>
      port: 5432
      dbname: jaffle_shop
      schema: analytics
      threads: 8

Configuring your target in Lightdash

Target contains information about your dbt connection to your warehouse. By default, we set this to be the same value as you have as the default in your profiles.yml file when you run lightdash deploy (if that’s how you created or recently deployed your project). If you want to update this, you can enter the target of your choice in the project settings (for example prod or analytics). Read more about dbt targets in the dbt docs.

How targets work with dbt macros

When Lightdash connects to your dbt project, it compiles a profiles.yml file based on the connection settings you configure in the Lightdash UI or the connection settings from your profiles.yml if you deployed your project from the CLI using lightdash deploy. This means that any dbt macros in your project that use target settings will be executed by Lightdash and resolved according to those settings.

Why use targets in macros?

You might want dbt to behave differently depending on the target/environment you’re running in. Common use cases include:

Using different schema naming conventions in dev vs. production
Configuring different warehouse sizes (small for dev, large for prod)
Adjusting query performance settings per environment
Loading different amounts of data (sample data in dev, full data in prod)
Applying different data retention policies

Let’s look at a common example: customizing schema names based on your target.

Example: Schema naming based on target

By default, dbt appends your custom schema names to your target schema. For example, if your target schema is analytics and you configure a model with schema='staging', dbt creates it in analytics_staging. This default behavior is useful in development because it prevents developers from overwriting each other’s work - each developer can have their own target schema like dbt_jane or dbt_john, resulting in separated schemas like dbt_jane_staging and dbt_john_staging. However, in production, you typically want cleaner schema names. Instead of analytics_staging, you just want staging. This is where customizing the generate_schema_name macro based on the target becomes helpful. Default behavior:

{% macro generate_schema_name(custom_schema_name, node) -%}
    {%- set default_schema = target.schema -%}
    {%- if custom_schema_name is none -%}
        {{ default_schema }}
    {%- else -%}
        {{ default_schema }}_{{ custom_schema_name | trim }}
    {%- endif -%}
{%- endmacro %}

Modified behavior for production:

{% macro generate_schema_name(custom_schema_name, node) -%}
    {%- set default_schema = target.schema -%}
    {%- if custom_schema_name is none -%}
        {{ default_schema }}
    {%- elif target.name == 'prod' -%}
        {{ custom_schema_name | trim }}
    {%- else -%}
        {{ default_schema }}_{{ custom_schema_name | trim }}
    {%- endif -%}
{%- endmacro %}

This macro says: “If we’re in production (target.name == 'prod'), use just the custom schema name. Otherwise, use the default behavior of prefixing it.” How this works in Lightdash: If your Lightdash connection settings specify:

Target name: prod
Dataset/Schema: analytics

And you have a model configured with:

{{ config(schema='staging') }}

Then Lightdash will create/query this model in the staging schema - not analytics_staging - because target.name == 'prod'. This gives you clean, professional schema names in production (staging, marts, reporting) while maintaining developer isolation in development environments (dbt_jane_staging, dbt_john_staging).

The generate_schema_name macro is a default dbt macro and its behavior can be overridden by adding a macro to dbt with the same name. If this macro is not showing up in your dbt project, it means that your project is using the default macro. It is a good idea to check your project for custom macros which can alter your project’s configurations.

Example: Warehouse configuration Similarly, if you have a macro that dynamically configures settings:

{% macro get_warehouse() %}
  {% if target.name == 'prod' %}
    {{ return('LARGE_WAREHOUSE') }}
  {% elif target.name == 'dev' %}
    {{ return('SMALL_WAREHOUSE') }}
  {% endif %}
{% endmacro %}

And use it in your models:

{{ config(snowflake_warehouse=get_warehouse()) }}

Lightdash will use the configuration that corresponds to the target name you’ve configured in your Lightdash connection settings.

Differences between dbt Core and dbt Cloud

In dbt Core, teams commonly use targets to manage different environments. In dbt Cloud, targets are replaced by environments, and custom logic such as if target.name = dev then... is commonly swapped out using env vars to become ‘if env_var = development then…’ or similar.

dbt environment variables

What are environment variables?

Environment variables allow you to set dynamic values that your dbt macros can reference. This is particularly useful if your dbt project uses environment variables to control behavior (common in dbt Cloud setups).

Configuring environment variables in Lightdash

This setting is found under Advanced configuration options

If your dbt project uses environment variables in your dbt profiles.yml file, you can add these to Lightdash in your project settings. For each environment variable, you’ll need to add the key + value pair for the item. You’ll normally find these values in a file called .env in your dbt project directory. For example, you might have something like:

profile:
  target: prod
  outputs:
    prod:
      type: postgres
      host: 127.0.0.1
      user: "{{ env_var('DBT_USER') }}"
      ....

Then a .env file like:

export DBT_USER="myspecialuser"

So, in Lightdash, you’d add a new environment variable and put key as DBT_USER and value as myspecialuser.

How environment variables work with dbt macros

When Lightdash compiles your dbt project, any environment variables you configure will be available to your dbt macros through the env_var() function. For example, if you have a macro that checks an environment variable:

{% macro get_schema() %}
  {% if env_var('DBT_ENV') == 'production' %}
    {{ return('prod_analytics') }}
  {% else %}
    {{ return('dev_analytics') }}
  {% endif %}
{% endmacro %}

You would set DBT_ENV=production as an environment variable in your Lightdash connection settings to ensure the macro resolves to prod_analytics. Key points about environment variables:

They’re available to all macros in your dbt project via the env_var() function
They’re particularly important if your dbt project was built for dbt Cloud, which relies heavily on environment variables
Make sure to set any environment variables your macros depend on in Lightdash to match your production dbt environment

Key takeaways

Lightdash compiles your dbt project using the target name and other settings you configure in the connection settings
Any macros that use target.name, environment variables, or other profile settings will resolve according to your Lightdash configuration
Make sure your Lightdash target name matches the target you use for your production dbt runs to ensure Lightdash behaves consistently with your dbt project
If you have custom macros that depend on target settings or environment variables, ensure those settings are configured correctly in your Lightdash connection settings

If you’re experiencing unexpected behavior with how Lightdash compiles your dbt project, check that your target name in Lightdash matches what your dbt macros expect, and verify any environment variables your macros depend on are properly set.

Documentation Index

​Syncing your dbt project to Lightdash

​Sync mechanisms

​lightdash deploy vs lightdash refresh

​Decision matrix

​Recommended setups

​Things to know

​How to trigger each sync

​Click “Refresh dbt” in Lightdash

​Push code from the CLI

​Set up continuous deployment

​dbt project settings

​dbt selector

​dbt targets

​What is a dbt target?

​Configuring your target in Lightdash

​How targets work with dbt macros

​Why use targets in macros?

​Example: Schema naming based on target

​Differences between dbt Core and dbt Cloud

​dbt environment variables

​What are environment variables?

​Configuring environment variables in Lightdash

​How environment variables work with dbt macros

​Key takeaways

Syncing your dbt project to Lightdash

Sync mechanisms

`lightdash deploy` vs `lightdash refresh`

Decision matrix

Recommended setups

Things to know

How to trigger each sync

Click “Refresh dbt” in Lightdash

Push code from the CLI

Set up continuous deployment

dbt project settings

dbt selector

dbt targets

What is a dbt target?

Configuring your target in Lightdash

How targets work with dbt macros

Why use targets in macros?

Example: Schema naming based on target

Differences between dbt Core and dbt Cloud

dbt environment variables

What are environment variables?

Configuring environment variables in Lightdash

How environment variables work with dbt macros

Key takeaways