feat(dbt-ingestion): added possibility to skip specific models #3340
Conversation
AndreasTA-AW
changed the title
|
Seems someone actually fixed the dbt prefix one day ago |
|
So I guess I should only add the model_name_pattern in that case |
AndreasTA-AW
changed the title
|
Is it possible to rerun the checks without pushing again? |
|
@AndreasTA-AW have you tried using |
|
I thought that one just checked the resource type, not the specifik name of a node. If that is the case it should work indeed. |
|
I double checked the code and it only looks at the resource_type, that is model, source, seed or test, but I would like the possibility to skip or only allow specifik models, seeds, test or sources, not the entire type |
|
Maybe it should be called name_pattern though, since it’s for all the types and not only models |
|
Right, (there is still some overlap with |
|
Ok, I'll rename it to node_name_pattern and hope the issue is done :) |
| owner = meta.get("owner") | ||
| if owner is None: | ||
| owner = manifest_node.get("config", {}).get("meta", {}).get("owner") |
There was a problem hiding this comment.
Are there multiple ways to provide owner?
One through
manifest -> meta -> owner and another through manifest -> config -> meta -> owner ?
There was a problem hiding this comment.
Yes, apparantly it's different for sources and models, which is a bit weird
Missed to use the tag prefix in last PR.
Also made it possible to filter out models that we would not want to push to DataHub. Common ones could be test seeds and such.
Updated the docs this time as well.
Checklist