slow to load transactions/activity when there are many transactions and the date range filter is applied

[AndySchroder]

Describe the feature

I have thousands of transactions. Zeus is very slow to load the transaction history even when I use the date range filter to only show one particular day that only has a few transactions. In that case, I would expect an instant result when the filter is active.

I'm using LND/REST.

Not sure if this is a Zeus issue, or the inability for LND to only respond a specified date range and Zeus just has to ask for all transactions all the time and then locally filter it.

[AndySchroder]

I see that lnrpc.ListPaymentsRequest (https://lightning.engineering/api-docs/api/lnd/lightning/list-payments#lnrpclistpaymentsrequest) and lnrpc.ListInvoiceRequest (https://lightning.engineering/api-docs/api/lnd/lightning/list-invoices) have creation_date_start and creation_date_end . Are these used by Zeus or is filtering done client side?

Also, I added some more filter requests for LND: lightningnetwork/lnd#7626 .

[AndySchroder]

#1322 was added. However, I noticed that ListPayments (https://lightning.engineering/api-docs/api/lnd/lightning/list-payments) does not return memos in lnrpc.Payment (https://lightning.engineering/api-docs/api/lnd/lightning/list-payments#lnrpcpayment). Are you computing those by decoding every invoice and then showing the memo that way? Wondering if it would be better to request that memos be added to the output in the lnrpc.Payment response in LND?

[kaloudis]

@AndySchroder do you use Tor to connect or clearnet? Have you tried doing any db compaction?

[AndySchroder]

I am using clearnet. The latency to the LND node is about 5ms.

Just to clarify the number of transactions we are talking about here:

$ lncli listpayments --count_total_payments|grep total_num_payments
    "total_num_payments": "30553"
$

I'm not sure how to do database compacting, but I don't know that that is the problem. Take a look at how long it takes LND to generate the data.

$ date;lncli listpayments>/dev/null ;date
Mon 26 Feb 2024 08:12:47 AM EST
Mon 26 Feb 2024 08:12:54 AM EST
$

Looks like it took 7 seconds for LND to fetch the data, but if I open up Activity on Zeus, it takes 30 seconds.

$ lncli listpayments|wc -c
108787889
$

Looks like it is over 100 megabytes of data that Zues is processing! Note, I've tested and it takes about 3 seconds to transfer that data over my network connection.

It seems like we have 7 seconds to generate the data and 3 seconds to transfer (assuming it doesn't begin transferring while it's still generating), so 20 more seconds for Zeus to read through the data.

Note, I do try to use Lightning Terminal's account feature to create a smaller account list, but Lightning Terminal in general is fairly buggy, so I can't use that in production.

[yanascz]

This bothers me as well. Would it make sense to implement infinite scrolling or pagination? It doesn't seem necessary to fetch all (matching) invoices at once when only about 12 fit the screen.

[AndySchroder]

I think infinite scrolling sounds like a good idea! That makes more sense than setting date filters. Zeus could pre-load the next transactions, maybe 20 at a time so it is not too slow.

Above I mentioned the use of creation_date_start. I just did some testing of this with fetching the last few months of transactions. It results in nearly a 400x reduction in data and a 10x reduction in time for LND to generate the data. I'd imagine that Zeus would be much faster too if it had to sort through less data.

$ lncli listpayments --creation_date_start 1700000000 |wc -c
279249
$

$ date +%s.%N;lncli listpayments --creation_date_start 1700000000 > /dev/null;date +%s.%N
1709836469.387800867
1709836470.071729589
$

[AndySchroder]

Actually, I probably should have tested with the --max_payments option instead. That makes more sense considering the infinite scrolling request and the fact that my nodes payment profile is arbitrary. That results in a 331 x reduction in data and an 86x speedup in time for LND to generate the data.

$ lncli listpayments --max_payments 100 |wc -c
328543
$

$ date +%s.%N;lncli listpayments --max_payments 100 > /dev/null;date +%s.%N
1709837130.631929307
1709837130.713093729
$

Offsetting doesn't seem to change the fetch time much.

$ date +%s.%N;lncli listpayments --max_payments 100 --index_offset 30460  > /dev/null;date +%s.%N
1709837553.720246664
1709837553.793363622
$

[AndySchroder]

Another thing if it is so fast fetching just a few transactions, Zeus should just show the most recent ones on the left screen (the one you press the Parthenon icon to get to) where you can see your balance. Then, just infinitely scroll down to see the rest of the transactions. Then there would be no need to have a dedicated Activity screen and it would help with the UX issues for in pending transactions in #1865 . I think that screen could be cleaned up a bit by compacting the lightning and on chain balances into one line. I think we could get at least 4 recent transactions on that screen before scrolling.

[AndySchroder]

Would like to add a bounty of 200,000 sat for fixing this issue.

[mjlamb]

I too find the "activity" screen very slow when loading and believe it may need re-thinking on how it is loading, happy to add 50,000 sats to the bounty pot.

[AndySchroder]

Also, it would be good to know if there is some other place we will get better results proposing our bounties for Zeus?

[AndySchroder]

Also, lightningnetwork/lnd#8530 is slightly related to this issue.

[myxmaster]

Is this related? #2465

[myxmaster]

While it is not the final fix for sure (we still need pagination and caching), this should improve performance significantly:
#2772

[huumn]

I make a bajillion zaps per day and get another bajillion incoming. I can't load my activity at all.

[kaloudis]

I make a bajillion zaps per day and get another bajillion incoming. I can't load my activity at all.

Can you try the new v0.9.5-beta? We made some improvements there.

[huumn]

Confirmed working in v0.9.5-beta2

[AndySchroder]

I make a bajillion zaps per day and get another bajillion incoming. I can't load my activity at all.

What is the unit conversion factor from bajillion zaps to zaps?