Fix!: Improve tracking of var dependencies in dbt models #5204
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This update improves the SQLMesh's ability to track variable dependencies for dbt models by adding a limited support for variable names that can't be determined statically.
If static analysis determines that the value of the first argument to the
{{ var() }}
macro call cannot be resolved, the Jinja string(s) will be rendered using a stub adapter implementation along with a{{ var() }}
macro override that captures the variable names passed into the macro during rendering.This approach is far from bulletproof, as rendering with the stub adapter may (and likely will) result in traversing code branches different from those that would be taken with the runtime adapter. At the same time, this approach is consistent with how dbt-core itself collects ref and source dependencies.
UPD: After further deliberation, we decided to unconditionally include ALL variables associated with a package if dynamic variable names are detected. This is due to the high risk of not capturing the correct variables during rendering, which could lead to incorrect runtime behavior in cases where dbt would have worked as expected.