You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When importing from GoCardless, if an account has transactions but they are all out of the filter range (or if they don't have transactions in the last 90 days at all), the data importer will exit with error code 1.
I've upgraded from 1.4.2 to 1.5.6 and this started happening (I vaguely remember this being a problem in the past that was fixed, so this might be a regression).
I'm aware that this might be intended behavior. If that's the case, feel free to dismiss this issue as I have a workaround for it (see additional information section).
Debug information
Debug information generated at 2024-10-17 10:01:58 Europe/Amsterdam for Firefly III Data Importer version 1.5.6.
System information
Item
Value
Version
1.5.6
System
PHP 8.3.12, Linux, fpm-fcgi
App information
Item
Value
Timezone
Europe/Amsterdam, Europe/Stockholm
Environment
production
Debug mode
false, cache 'file'
Log level
debug, stack
Display errors
Off, ALL errors
BCscale
12
Trusted proxies
User information
Item
Value
User agent
Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0
Expected behaviour
If there are no transactions, I would still expect the exit code to be 0 if there was no actual error talking to GoCardless.
Steps to reproduce
Probably not very useful, but this is my JSON config for that specific account:
If there are no transactions in the last 5 days, exit code of the importer is 1.
Additional info
Log (slightly redacted):
[2024-10-16 13:45:55] production.DEBUG: ImportFile: directory "/persist/firefly-auto-import-configs"
[2024-10-16 13:45:55] production.DEBUG: ImportFile: file "REDACTED_BANK-nordigen.json"
[2024-10-16 13:45:55] production.DEBUG: ImportFile: importable "/persist/firefly-auto-import-configs/REDACTED_BANK-nordigen.json"
[2024-10-16 13:45:55] production.DEBUG: ImportFile: JSON "/persist/firefly-auto-import-configs/REDACTED_BANK-nordigen.json"
[2024-10-16 13:45:55] production.DEBUG: Now in updateDateRange()
[2024-10-16 13:45:55] production.DEBUG: Range is partial, after is NULL, dateNotBefore will be calculated.
[2024-10-16 13:45:55] production.DEBUG: dateNotBefore is now "2024-10-11"
Going to convert from file /persist/firefly-auto-import-configs/REDACTED_BANK-nordigen.json using configuration /persist/firefly-auto-import-configs/REDACTED_BANK-nordigen.json and flow "nordigen".
[2024-10-16 13:45:55] production.DEBUG: Now in App\Console\AutoImports::startConversion
[2024-10-16 13:45:55] production.DEBUG: Going to generate conversion routine identifier.
[2024-10-16 13:45:55] production.DEBUG: Attempt #1 results in "conversion-NL8J7w38Xt2s"
[2024-10-16 13:45:55] production.INFO: Job identifier is "conversion-NL8J7w38Xt2s"
[2024-10-16 13:45:55] production.DEBUG: Now in startOrFindConversion(conversion-NL8J7w38Xt2s)
[2024-10-16 13:45:55] production.DEBUG: File does not exist or error, create a new one.
[2024-10-16 13:45:55] production.DEBUG: Return status. {"status":"waiting_to_start","errors":[],"warnings":[],"messages":[]}
[2024-10-16 13:45:55] production.DEBUG: Now in setConversionStatus(conv_running)
[2024-10-16 13:45:55] production.DEBUG: Found "conversion-NL8J7w38Xt2s" in the session
[2024-10-16 13:45:55] production.DEBUG: Now in startOrFindConversion(conversion-NL8J7w38Xt2s)
[2024-10-16 13:45:55] production.DEBUG: Conversion status is "waiting_to_start"
[2024-10-16 13:45:55] production.DEBUG: Now in storeConversionStatus(conversion-NL8J7w38Xt2s): conv_running
[2024-10-16 13:45:55] production.DEBUG: Messages: 0, warnings: 0, errors: 0
[2024-10-16 13:45:55] production.DEBUG: Set identifier to "conversion-NL8J7w38Xt2s"
[2024-10-16 13:45:55] production.DEBUG: Set identifier to "conversion-NL8J7w38Xt2s"
[2024-10-16 13:45:55] production.DEBUG: Now in App\Services\Nordigen\Conversion\RoutineManager::start
[2024-10-16 13:45:55] production.DEBUG: The GoCardless API URL is https://bankaccountdata.gocardless.com
[2024-10-16 13:45:55] production.DEBUG: Call transaction processor download.
[2024-10-16 13:45:55] production.DEBUG: Now in App\Services\Nordigen\Conversion\Routine\TransactionProcessor::download
[2024-10-16 13:45:55] production.DEBUG: Found 1 accounts to download from.
[2024-10-16 13:45:55] production.DEBUG: Going to download transactions for account #0 "REDACTED"
[2024-10-16 13:45:55] production.DEBUG: Now in App\Services\Nordigen\Services\AccountInformationCollector::collectInformation
[2024-10-16 13:45:55] production.DEBUG: Get account details is DISABLED.
[2024-10-16 13:45:55] production.DEBUG: Get account balance is DISABLED.
[2024-10-16 13:45:55] production.DEBUG: Now in App\Services\Nordigen\Services\AccountInformationCollector::getBasicDetails(REDACTED)
[2024-10-16 13:45:55] production.DEBUG: Request parameters will be set to:
[2024-10-16 13:45:55] production.DEBUG: authenticatedGet(https://bankaccountdata.gocardless.com/api/v2/accounts/REDACTED)
[2024-10-16 13:45:55] production.DEBUG: Rate limit: 100
[2024-10-16 13:45:55] production.DEBUG: Rate limit remaining: 82
[2024-10-16 13:45:55] production.DEBUG: Rate limit reset: 26
[2024-10-16 13:45:55] production.DEBUG: Now in pauseForRateLimit
[2024-10-16 13:45:55] production.DEBUG: Rate limit: 82 requests left, and 26s before the limit resets.
[2024-10-16 13:45:55] production.DEBUG: Save the account success limits? NO
[2024-10-16 13:45:55] production.DEBUG: Response for basic information request: {"id":"REDACTED","created":"REDACTED","last_accessed":"REDACTED","iban":"REDACTED","institution_id":"REDACTED","status":"READY","owner_name":""}
[2024-10-16 13:45:55] production.DEBUG: Set new IBAN from basic details.
[2024-10-16 13:45:55] production.DEBUG: Done downloading information for debug purposes.
[2024-10-16 13:45:55] production.DEBUG: Request parameters will be set to:
[2024-10-16 13:45:55] production.DEBUG: authenticatedGet(https://bankaccountdata.gocardless.com/api/v2/accounts/REDACTED/transactions/)
[2024-10-16 13:45:57] production.DEBUG: Rate limit: 100
[2024-10-16 13:45:57] production.DEBUG: Rate limit remaining: 81
[2024-10-16 13:45:57] production.DEBUG: Rate limit reset: 25
[2024-10-16 13:45:57] production.DEBUG: Account success rate limit: 10
[2024-10-16 13:45:57] production.DEBUG: Account success rate limit remaining: 8
[2024-10-16 13:45:57] production.DEBUG: Account success rate limit reset: 40392
[2024-10-16 13:45:57] production.DEBUG: Now in pauseForRateLimit
[2024-10-16 13:45:57] production.DEBUG: Rate limit: 81 requests left, and 25s before the limit resets.
[2024-10-16 13:45:57] production.DEBUG: Save the account success limits? YES
[2024-10-16 13:45:57] production.DEBUG: Account success limit: 8 requests left, and 11h13m12s before the limit resets.
[2024-10-16 13:45:57] production.DEBUG: Downloaded [12:12] transactions
[2024-10-16 13:45:57] production.DEBUG: Nordigen transaction from array {"bookingDate":"2024-10-04","valueDate":"2024-10-04","transactionAmount":{"amount":"REDACTED","currency":"EUR"},"remittanceInformationUnstructured":"REDACTED","internalTransactionId":"REDACTED","key":"booked"}
[2024-10-16 13:45:57] production.DEBUG: Downloaded transaction with ID "REDACTED"
[2024-10-16 13:45:57] production.DEBUG: Nordigen transaction from array REDACTED
[2024-10-16 13:45:57] production.DEBUG: Downloaded transaction with ID "REDACTED"
[2024-10-16 13:45:57] production.DEBUG: Nordigen transaction from array REDACTED
[2024-10-16 13:45:57] production.DEBUG: Downloaded transaction with ID "REDACTED"
[2024-10-16 13:45:57] production.DEBUG: Nordigen transaction from array REDACTED
[2024-10-16 13:45:57] production.DEBUG: Downloaded transaction with ID "REDACTED"
[2024-10-16 13:45:57] production.DEBUG: Nordigen transaction from array REDACTED
[2024-10-16 13:45:57] production.DEBUG: Downloaded transaction with ID "REDACTED"
[2024-10-16 13:45:57] production.DEBUG: Nordigen transaction from array REDACTED
[2024-10-16 13:45:57] production.DEBUG: Downloaded transaction with ID "REDACTED"
[2024-10-16 13:45:57] production.DEBUG: Nordigen transaction from array REDACTED
[2024-10-16 13:45:57] production.DEBUG: Downloaded transaction with ID "REDACTED"
[2024-10-16 13:45:57] production.DEBUG: Nordigen transaction from array REDACTED
[2024-10-16 13:45:57] production.DEBUG: Downloaded transaction with ID "REDACTED"
[2024-10-16 13:45:57] production.DEBUG: Nordigen transaction from array REDACTED
[2024-10-16 13:45:57] production.DEBUG: Downloaded transaction with ID "REDACTED"
[2024-10-16 13:45:57] production.DEBUG: Nordigen transaction from array REDACTED
[2024-10-16 13:45:57] production.DEBUG: Downloaded transaction with ID "REDACTED"
[2024-10-16 13:45:57] production.DEBUG: Nordigen transaction from array REDACTED
[2024-10-16 13:45:57] production.DEBUG: Downloaded transaction with ID "REDACTED"
[2024-10-16 13:45:57] production.DEBUG: Nordigen transaction from array {"bookingDate":"2024-08-05","valueDate":"2024-08-05","transactionAmount":{"amount":"REDACTED","currency":"EUR"},"remittanceInformationUnstructured":"REDACTED","internalTransactionId":"REDACTED","key":"booked"}
[2024-10-16 13:45:57] production.DEBUG: Downloaded transaction with ID "REDACTED"
[2024-10-16 13:45:57] production.DEBUG: GetTransactionsResponse: count 12 transaction(s)
[2024-10-16 13:45:57] production.INFO: Going to filter downloaded transactions. Original set length is 12
[2024-10-16 13:45:57] production.INFO: Will not grab transactions before "2024-10-11 00:00:00"
[2024-10-16 13:45:57] production.DEBUG: Returning book date
[2024-10-16 13:45:57] production.DEBUG: Skip transaction because "2024-10-04 00:00:00" is before "2024-10-11 00:00:00".
[2024-10-16 13:45:57] production.DEBUG: Returning book date
[2024-10-16 13:45:57] production.DEBUG: Skip transaction because "REDACTED" is before "2024-10-11 00:00:00".
[2024-10-16 13:45:57] production.DEBUG: Returning book date
[2024-10-16 13:45:57] production.DEBUG: Skip transaction because "REDACTED" is before "2024-10-11 00:00:00".
[2024-10-16 13:45:57] production.DEBUG: Returning book date
[2024-10-16 13:45:57] production.DEBUG: Skip transaction because "REDACTED" is before "2024-10-11 00:00:00".
[2024-10-16 13:45:57] production.DEBUG: Returning book date
[2024-10-16 13:45:57] production.DEBUG: Skip transaction because "REDACTED" is before "2024-10-11 00:00:00".
[2024-10-16 13:45:57] production.DEBUG: Returning book date
[2024-10-16 13:45:57] production.DEBUG: Skip transaction because "REDACTED" is before "2024-10-11 00:00:00".
[2024-10-16 13:45:57] production.DEBUG: Returning book date
[2024-10-16 13:45:57] production.DEBUG: Skip transaction because "REDACTED" is before "2024-10-11 00:00:00".
[2024-10-16 13:45:57] production.DEBUG: Returning book date
[2024-10-16 13:45:57] production.DEBUG: Skip transaction because "REDACTED" is before "2024-10-11 00:00:00".
[2024-10-16 13:45:57] production.DEBUG: Returning book date
[2024-10-16 13:45:57] production.DEBUG: Skip transaction because "REDACTED" is before "2024-10-11 00:00:00".
[2024-10-16 13:45:57] production.DEBUG: Returning book date
[2024-10-16 13:45:57] production.DEBUG: Skip transaction because "REDACTED" is before "2024-10-11 00:00:00".
[2024-10-16 13:45:57] production.DEBUG: Returning book date
[2024-10-16 13:45:57] production.DEBUG: Skip transaction because "REDACTED" is before "2024-10-11 00:00:00".
[2024-10-16 13:45:57] production.DEBUG: Returning book date
[2024-10-16 13:45:57] production.DEBUG: Skip transaction because "2024-08-05 00:00:00" is before "2024-10-11 00:00:00".
[2024-10-16 13:45:57] production.INFO: After filtering, set is 0 transaction(s)
[2024-10-16 13:45:57] production.DEBUG: Done downloading transactions for account 0 "REDACTED"
[2024-10-16 13:45:57] production.DEBUG: Done with download
[2024-10-16 13:45:57] production.DEBUG: Rate limit for account REDACTED: 8 request(s) left, 40392 second(s)
[2024-10-16 13:45:57] production.DEBUG: Generating Firefly III transactions.
[2024-10-16 13:45:57] production.DEBUG: Nordigen: Defer account search to trait.
[2024-10-16 13:45:57] production.DEBUG: Now in collectAllTargetAccounts()
[2024-10-16 13:45:57] production.DEBUG: Now in collectAccounts("asset")
[2024-10-16 13:45:57] production.DEBUG: No access token in hasAccessToken() session, will return config variable.
[2024-10-16 13:45:57] production.DEBUG: No base url in getBaseUrl() session, will return config variable.
[2024-10-16 13:45:58] production.DEBUG: Found 16 accounts of type "asset"
[2024-10-16 13:45:58] production.DEBUG: Processing account #7 ("REDACTED") with type "asset"
[2024-10-16 13:45:58] production.DEBUG: Processing account #11 ("REDACTED") with type "asset"
[2024-10-16 13:45:58] production.DEBUG: Processing account #9 ("REDACTED") with type "asset"
[2024-10-16 13:45:58] production.DEBUG: Processing account #4 ("REDACTED") with type "asset"
[2024-10-16 13:45:58] production.DEBUG: Processing account #13 ("REDACTED") with type "asset"
[2024-10-16 13:45:58] production.DEBUG: Processing account #15 ("REDACTED") with type "asset"
[2024-10-16 13:45:58] production.DEBUG: Processing account #5 ("REDACTED") with type "asset"
[2024-10-16 13:45:58] production.DEBUG: Collected account IBAN "REDACTED" (asset) under ID #5
[2024-10-16 13:45:58] production.DEBUG: Processing account #82 ("REDACTED") with type "asset"
[2024-10-16 13:45:58] production.DEBUG: Processing account #147 ("REDACTED") with type "asset"
[2024-10-16 13:45:58] production.DEBUG: Processing account #1 ("REDACTED") with type "asset"
[2024-10-16 13:45:58] production.DEBUG: Processing account #3 ("REDACTED") with type "asset"
[2024-10-16 13:45:58] production.DEBUG: Processing account #29 ("REDACTED") with type "asset"
[2024-10-16 13:45:58] production.DEBUG: Collected account IBAN "REDACTED" (asset) under ID #29
[2024-10-16 13:45:58] production.DEBUG: Processing account #33 ("REDACTED") with type "asset"
[2024-10-16 13:45:58] production.DEBUG: Processing account #55 ("REDACTED") with type "asset"
[2024-10-16 13:45:58] production.DEBUG: Processing account #128 ("REDACTED") with type "asset"
[2024-10-16 13:45:58] production.DEBUG: Collected account IBAN "REDACTED" (asset) under ID #128
[2024-10-16 13:45:58] production.DEBUG: Processing account #129 ("REDACTED") with type "asset"
[2024-10-16 13:45:58] production.DEBUG: Collected account IBAN "REDACTED" (asset) under ID #129
[2024-10-16 13:45:58] production.DEBUG: Collected 16 accounts of type "asset"
[2024-10-16 13:45:58] production.DEBUG: Now in collectAccounts("liabilities")
[2024-10-16 13:45:58] production.DEBUG: No access token in hasAccessToken() session, will return config variable.
[2024-10-16 13:45:58] production.DEBUG: No base url in getBaseUrl() session, will return config variable.
[2024-10-16 13:45:58] production.DEBUG: Found 0 accounts of type "liabilities"
[2024-10-16 13:45:58] production.DEBUG: Collected 0 accounts of type "liabilities"
[2024-10-16 13:45:58] production.DEBUG: Nordigen: Collected 4 accounts.
[2024-10-16 13:45:58] production.DEBUG: Add message about rate limits.
[2024-10-16 13:45:58] production.DEBUG: Add message about rate limits for account 1.
[2024-10-16 13:45:58] production.DEBUG: Found NO Firefly III account to report on, will not report rate limit.
[2024-10-16 13:45:58] production.WARNING: Downloaded nothing, will return nothing.
[2024-10-16 13:45:58] production.ERROR: [c] Add error to index #0: No transactions were downloaded from GoCardless. You may be rate limited or something else went wrong.
[2024-10-16 13:45:58] production.DEBUG: Add error on index #0 (line no. 1): No transactions were downloaded from GoCardless. You may be rate limited or something else went wrong.
[2024-10-16 13:45:58] production.DEBUG: Now in storeConversionStatus(conversion-NL8J7w38Xt2s): conv_running
[2024-10-16 13:45:58] production.DEBUG: Messages: 0, warnings: 0, errors: 1
[2024-10-16 13:45:58] production.ERROR: [a] Zero transactions!
[2024-10-16 13:45:58] production.DEBUG: Now in setConversionStatus(conv_done)
[2024-10-16 13:45:58] production.DEBUG: Found "conversion-NL8J7w38Xt2s" in the session
[2024-10-16 13:45:58] production.DEBUG: Now in startOrFindConversion(conversion-NL8J7w38Xt2s)
[2024-10-16 13:45:58] production.DEBUG: Conversion status is "conv_running"
[2024-10-16 13:45:58] production.DEBUG: Now in storeConversionStatus(conversion-NL8J7w38Xt2s): conv_done
[2024-10-16 13:45:58] production.DEBUG: Messages: 0, warnings: 0, errors: 1
Conversion index (error) 0: No transactions were downloaded from GoCardless. You may be rate limited or something else went wrong.
Too many errors in the data conversion (1), exit.
[2024-10-16 13:45:58] production.DEBUG: Created event ImportedTransactions with filtering (2)
[2024-10-16 13:45:58] production.DEBUG: Array contains 0 message(s)
[2024-10-16 13:45:58] production.DEBUG: Array contains 0 warning(s)
[2024-10-16 13:45:58] production.DEBUG: Array contains 1 error(s)
[2024-10-16 13:45:58] production.DEBUG: Now in sendReportOverMail
[2024-10-16 13:45:58] production.INFO: Will send report message.
[2024-10-16 13:45:58] production.DEBUG: Messages count: 0
[2024-10-16 13:45:58] production.DEBUG: Warnings count: 0
[2024-10-16 13:45:58] production.DEBUG: Errors count : 1
[2024-10-16 13:45:58] production.DEBUG: If no error below this line, mail was sent!
[2024-10-16 13:45:58] production.DEBUG: If no error above this line, mail was sent!
[2024-10-16 13:45:58] production.ERROR: Could not complete import from file "REDACTED_BANK-nordigen.json".
[2024-10-16 13:45:58] production.ERROR: Too many errors in the data conversion.
I remember somebody wanted specifically to exit 1 when no no transactions were found or imported or whatever so that's what I did. It made sense at the time.
I could change to exit code 2 or something like that, I'll think about it!
I can see this being useful in certain situations yeah.
For context, I'm running this every day across all my accounts and I have monitoring in place for when services exit with error (non-zero), so it was triggering an alert 😅
For checking if transactions were imported or not, I have the email reports.
Thanks for the quick response, let me know if you need anything else from my side
The importer:import command and importer:auto-import will exit 73 when nothing was imported. The importer:auto-import command may also exit 1 in the same situation, since it can process multiple files (and another file may have had an import that was perfectly fine, or had another error).
Support guidelines
I've found a bug and checked that ...
Description
When importing from GoCardless, if an account has transactions but they are all out of the filter range (or if they don't have transactions in the last 90 days at all), the data importer will exit with error code 1.
I've upgraded from 1.4.2 to 1.5.6 and this started happening (I vaguely remember this being a problem in the past that was fixed, so this might be a regression).
I'm aware that this might be intended behavior. If that's the case, feel free to dismiss this issue as I have a workaround for it (see additional information section).
Debug information
Debug information generated at 2024-10-17 10:01:58 Europe/Amsterdam for Firefly III Data Importer version 1.5.6.
Expected behaviour
If there are no transactions, I would still expect the exit code to be 0 if there was no actual error talking to GoCardless.
Steps to reproduce
Probably not very useful, but this is my JSON config for that specific account:
If there are no transactions in the last 5 days, exit code of the importer is 1.
Additional info
Log (slightly redacted):
I've fixed this on my side by making the no transactions error a warning instead: diogotcorreia/dotfiles@399f824#diff-19cd1f20eb3f5b530f5ad59217e4f13e68b6fced9575624b2787b4fcb725b795 (file
0001-no-transactions-warning-instead-of-error.diff
).The text was updated successfully, but these errors were encountered: