- Incorporated the new
fivetran_utils.drop_schemas_automation
macro into the end of each Buildkite integration test job. - Updated the pull request templates.
PR #66 applies the following changes:
- Adds
transaction_id
andtransaction_line_id
to the Netsuite (1) income statement and balance sheet models. These fields were already present in the Netsuite2 versions of these models. These columns are included in newly added Primary Key tests on the Netsuite (1) income statement and balance sheet models.
Note: Ensure that neither of these columns are included in your
balance_sheet_transaction_detail_columns
orincome_statement_transaction_detail_columns
variables.
- Aligns join types in Netsuite end models with the joins in Netsuite2 end models.
- Adds new account type <> account category mappings in the
int_netsuite__transactions_with_converted_amounts
/int_netsuite2__tran_with_converted_amounts
model.Prepaid Expense
account types are treated asDeferred Expense
accounts.Non Posting
andStatistical
account types will be placed in a newOther
category.
- Adds an
accepted_values
test on the transaction detail end models that will raise a warning if unexpected account types are encountered. - Adds a DECISIONLOG.
- Updates README to include the
netsuite2__using_jobs
variable. - Adds uniqueness and not-null tests to the Netsuite (1) income statement and balance sheet models. These tests were already present in the Netsuite2 models.
- Now introducing...Databricks compatibility 🧱 (PR #61)
- Adjustment to add persist pass_through_columns macro to Netsuite1 models (#60)
PR #53 includes the following breaking changes:
- Dispatch update for dbt-utils to dbt-core cross-db macros migration. Specifically
{{ dbt_utils.<macro> }}
have been updated to{{ dbt.<macro> }}
for the below macros:any_value
bool_or
cast_bool_to_text
concat
date_trunc
dateadd
datediff
escape_single_quotes
except
hash
intersect
last_day
length
listagg
position
replace
right
safe_cast
split_part
string_literal
type_bigint
type_float
type_int
type_numeric
type_string
type_timestamp
array_append
array_concat
array_construct
- For
current_timestamp
andcurrent_timestamp_in_utc
macros, the dispatch AND the macro names have been updated to the below, respectively:dbt.current_timestamp_backcompat
dbt.current_timestamp_in_utc_backcompat
packages.yml
has been updated to reflect new defaultfivetran/fivetran_utils
version, previously[">=0.3.0", "<0.4.0"]
now[">=0.4.0", "<0.5.0"]
.
- Adjustment within the
int_netsuite2_tran_lines_w_accounting_period
model to correctly filter only posting accounts. Previously this filter filtered for only non-posting accounts. In order to replicate a true income statement, the posting accounts should only be included downstream. (#56)
PR #48 includes the following updates to the dbt_netsuite package:
- Introduces the
netsuite2__multibook_accounting_enabled
andnetsuite2__using_vendor_categories
variables to disable their related source tables and downstream models.netsuite2__multibook_accounting_enabled
isTrue
by default. Set it toFalse
if you do not use the Multi-Book Accounting feature in Netsuite and/or do not have theaccountingbook
andaccountingbooksubsidiaries
source tables.netsuite2__using_vendor_categories
isTrue
by default. Set it toFalse
if you do not categorize vendors in Netsuite and/or do not have thevendorcategory
source table.
- Fixes the grain at which the
netsuite__transaction_details
model is tested (Netsuite.com users only).
- Properly applies new passthrough column logic to allow for the use of
alias
andtransform_sql
(see v0.6.0 below). (#43)
🎉 Netsuite2 Compatibility 🎉 PR #41 includes the following update to the dbt_netsuite package:
- The declaration of passthrough variables within your root
dbt_project.yml
has changed. To allow for more flexibility and better tracking of passthrough columns, you will now want to define passthrough columns in the following format:
This applies to all passthrough columns within the
dbt_netsuite
package and not just thecustomers_pass_through_columns
example.
vars:
customers_pass_through_columns:
- name: "my_field_to_include" # Required: Name of the field within the source.
alias: "field_alias" # Optional: If you wish to alias the field within the staging model.
transform_sql: "cast(field_alias as string)" # Optional: If you wish to define the datatype or apply a light transformation.
- Addition of the
netsuite_data_model
variable. This variable may either benetsuite
(the original Netsuite.com connector endpoint) ornetsuite2
(the new Netsuite2 connector endpoint).- The variable is set to
netsuite
by default. If you wish to run the data models for the Netsuite2 connector, you may simply change the variable within your root dbt_project.yml tonetsuite2
.
- The variable is set to
- Postgres compatibility!
- Added identifier variables to each Netsuite.com and Netsuite2 source to enable dynamic source-table adjustments.
- Applied schema level tests to each Netsuite2 end model to ensure data validation.
- README updates for easier navigation and package use.
🎉 dbt v1.0.0 Compatibility 🎉
- Adjusts the
require-dbt-version
to now be within the range [">=1.0.0", "<2.0.0"]. Additionally, the package has been updated for dbt v1.0.0 compatibility. If you are using a dbt version <1.0.0, you will need to upgrade in order to leverage the latest version of the package.- For help upgrading your package, I recommend reviewing this GitHub repo's Release Notes on what changes have been implemented since your last upgrade.
- For help upgrading your dbt project to dbt v1.0.0, I recommend reviewing dbt-labs upgrading to 1.0.0 docs for more details on what changes must be made.
- Upgrades the package dependency to refer to the latest
dbt_netsuite_source
. Additionally, the latestdbt_netsuite_source
package has a dependency on the latestdbt_fivetran_utils
. Further, the latestdbt_fivetran_utils
package also has a dependency ondbt_utils
[">=0.8.0", "<0.9.0"].- Please note, if you are installing a version of
dbt_utils
in yourpackages.yml
that is not in the range above then you will encounter a package dependency error.
- Please note, if you are installing a version of
Refer to the relevant release notes on the Github repository for specific details for the previous releases. Thank you!