Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

handle missing packages exception #174

Merged
merged 2 commits into from
Sep 19, 2023
Merged

handle missing packages exception #174

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a89b7a7
handle missing packages exception
dave-connors-3 Sep 16, 2023
e08e3bf
move exception handling to invoke method
dave-connors-3 Sep 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion dbt_meshify/dbt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
from dbt.cli.main import dbtRunner
from dbt.contracts.graph.manifest import Manifest
from dbt.contracts.results import CatalogArtifact
from dbt.exceptions import UninstalledPackagesFoundError
from loguru import logger


Expand All @@ -27,7 +28,12 @@ def invoke(

def parse(self, directory: os.PathLike):
logger.info("Executing dbt parse...")
return self.invoke(directory, ["--quiet", "parse"])
try:
return self.invoke(directory, ["--quiet", "parse"])
except UninstalledPackagesFoundError:
logger.debug("Project missing packages, installing...")
self.invoke(directory, ["deps"])
return self.invoke(directory, ["--quiet", "parse"])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very clever! 🤔 Does this exception ever get raised for other Dbt methods? If so, it may make sense to wrap/decorate invoke() directly to handle this exception for all dbt calls. Thoughts, @dave-connors-3?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah great point! I think it's around for any submitted command when packages aren't installed, but I put it in parse since it's always the first method we call. I can add it up a level to invoke to be safer!


def ls(
self,
Expand Down
16 changes: 15 additions & 1 deletion tests/unit/test_dbt_projects.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,15 @@
# TODO -- we don't really have a test suite for DbtProjects and their properties
import shutil
from pathlib import Path

from dbt_meshify.dbt_projects import DbtProject

split_project_path = "test-projects/split/split_proj"


class TestDbtProject:
def test_missing_packages(self):
package_path = Path(split_project_path) / "dbt_packages"
if package_path.exists():
shutil.rmtree(package_path)
project = DbtProject.from_directory(split_project_path, read_catalog=True)
assert project.manifest is not None