-
Notifications
You must be signed in to change notification settings - Fork 132
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Refactored dbt utility for modularity
- Loading branch information
1 parent
941c6a4
commit 636e109
Showing
7 changed files
with
552 additions
and
333 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
"""Utility for running and logging output from dbt commands | ||
Enable this utility by installing parsons with a dbt extra: | ||
`pip install parsons[dbt-redshift]` | ||
or `pip install parsons[dbt-postgres]` | ||
or `pip install parsons[dbt-snowflake]` | ||
or `pip install parsons[dbt-bigquery]` | ||
To run dbt commands, you will need to have a dbt project directory | ||
somewhere on the local filesystem. | ||
The dbt command will inherit environment variables from the python | ||
process shell, so if your dbt profiles.yml file uses environment | ||
variables, ensure those are set in python or the parent shell before | ||
running this dbt utility. | ||
Logging is handled separately from the dbt run itself. The | ||
dbtRunner.run method returns a dbtCommandResult object which can be | ||
passed to a child class of dbtLogger for logging to stdout, slack, | ||
etc. | ||
Parsons provides a few example dbtLogger child classes, but for | ||
best results, design your own! | ||
Example usage: | ||
``` | ||
from parsons.utilities.dbt import ( | ||
run_dbt_commands, | ||
dbtLoggerSlack, | ||
dbtLoggerPython | ||
) | ||
run_dbt_commands( | ||
commands=['run', 'test'], | ||
dbt_project_directory='/home/ubuntu/code/dbt_project/', | ||
loggers=[ | ||
dbtLoggerPython, | ||
dbtLoggerSlack(slack_webhook=os.environ['SLACK_WEBHOOK']) | ||
] | ||
) | ||
``` | ||
""" | ||
|
||
from parsons.utilities.dbt.dbt import run_dbt_commands | ||
from parsons.utilities.dbt.logging import ( | ||
dbtLoggerMarkdown, | ||
dbtLoggerSlack, | ||
dbtLoggerStdout, | ||
dbtLoggerPython, | ||
) | ||
|
||
__all__ = [ | ||
"run_dbt_commands", | ||
"dbtLoggerMarkdown", | ||
"dbtLoggerSlack", | ||
"dbtLoggerStdout", | ||
"dbtLoggerPython", | ||
] |
Oops, something went wrong.