A quick explainer of the "dbt was unable to infer all dependencies for the model" error #dbt

README.md

A quick explainer of the "dbt was unable to infer all dependencies for the model" error

Official docs: https://docs.getdbt.com/reference/dbt-jinja-functions/ref#forcing-dependencies

If we have a model that uses a macro and within that macro is a ref() to another model like so:

-- models/foo.sql
select 1 id

-- models/bar.sql
{{ some_macro() }}

-- macros/some_macro.sql
{% macro some_macro() %}
    {% if execute %}
        select * from {{ ref("foo") }}
    {% else %}
    {% endif %}
{% endmacro %}

If we try running the above example:

$ dbt run

00:45:25  Running with dbt=1.6.9
00:45:25  Registered adapter: postgres=1.6.9
00:45:26  Found 2 models, 0 sources, 0 exposures, 0 metrics, 353 macros, 0 groups, 0 semantic models
00:45:26  
00:45:26  Concurrency: 1 threads (target='pg-local')
00:45:26  
00:45:26  1 of 2 START sql table model public.bar ........................................ [RUN]
00:45:26  1 of 2 ERROR creating sql table model public.bar ............................... [ERROR in 0.02s]
00:45:26  2 of 2 START sql table model public.foo ........................................ [RUN]
00:45:26  2 of 2 OK created sql table model public.foo ................................... [SELECT 1 in 0.15s]
00:45:26  
00:45:26  Finished running 2 table models in 0 hours 0 minutes and 0.37 seconds (0.37s).
00:45:26  
00:45:26  Completed with 1 error and 0 warnings:
00:45:26  
00:45:26    Compilation Error in model bar (models/bar.sql)
  dbt was unable to infer all dependencies for the model "bar".
  This typically happens when ref() is placed within a conditional block.
  
  To fix this, add the following hint to the top of the model "bar":
  
  -- depends_on: {{ ref('foo') }}
  
  > in macro some_macro (macros/some_macro.sql)
  > called by model bar (models/bar.sql)
00:45:26  
00:45:26  Done. PASS=1 WARN=0 ERROR=1 SKIP=0 TOTAL=2

We can actually see that dbt tried to run bar first as well cause it couldn't establish the dependency chain.

dbt is not able to infer the dependencies - i.e. in model bar, it calls the macro some_macro() - however some_macro() itself depends on another model foo. The dependency chain should be:

┌────┐   ┌────┐
│foo │──▶│bar │
└────┘   └────┘

Since bar should ALWAYS run after foo.

But because we have "hidden" that dependency via the macro and an if execute gate, we need to explicitly tell dbt that model bar has to wait for foo to run first by adding an explicit depends_on to bar:

-- models/bar.sql
-- depends_on: {{ ref("foo") }}
{{ some_macro() }}

$ dbt run
00:48:15  Running with dbt=1.6.9
00:48:15  Registered adapter: postgres=1.6.9
00:48:16  Found 2 models, 0 sources, 0 exposures, 0 metrics, 353 macros, 0 groups, 0 semantic models
00:48:16  
00:48:16  Concurrency: 1 threads (target='pg-local')
00:48:16  
00:48:16  1 of 2 START sql table model public.foo ........................................ [RUN]
00:48:16  1 of 2 OK created sql table model public.foo ................................... [SELECT 1 in 0.17s]
00:48:16  2 of 2 START sql table model public.bar ........................................ [RUN]
00:48:16  2 of 2 OK created sql table model public.bar ................................... [SELECT 1 in 0.08s]
00:48:16  
00:48:16  Finished running 2 table models in 0 hours 0 minutes and 0.42 seconds (0.42s).
00:48:16  
00:48:16  Completed successfully
00:48:16  
00:48:16  Done. PASS=2 WARN=0 ERROR=0 SKIP=0 TOTAL=2