-
Notifications
You must be signed in to change notification settings - Fork 43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support sync access by default #18
Comments
Alternatively, if you're concerned about parsing large ledger files, could we switch to a callback model as recommend by David Cheney? https://dave.cheney.net/practical-go/presentations/qcon-china.html#_leave_concurrency_to_the_caller |
I don't understand why the internal methods used within the package would matter to the caller. If you want to handle the ledger as a stream of transactions, processing them as they are parsed, call ParseLedgerAsync(). If you're advocating for a public API that allows for the caller to pass a callback function that is executed per transaction, I'm going to respectfully decline. |
I realize I can call ParseLedger as a synchronous blocking API, my problem is that internally it’s asynchronous, and the memory allocations for those channels together with the context switching running a separate go-routine are a performance bottleneck for my application. I need a way to opt out of the goroutine and channels. For context, benchmarking my application, ParseLedger takes 12% of total execution time (for a 15ms process). It’s not because ledger is inherently slow - I’ve looked at your benchmarks - it’s extra time in the scheduler and garbage collector due to, for me, unnecessary concurrency |
Interesting. Okay sure, go for it. |
LGTM |
ParseLedger() is just a wrapper for ParseLedgerAsync, which means there is no public synchronous parsing API. If my program is going to be blocked anyway waiting for the results of a ledger parse, I don't want to have the overhead of channels and scheduler context switches on top of that.
Could we switch it around - so ParseLedger is where the work actually happens, and ParseLedgerAsync wraps it in a goroutine and channels? I would be happy to submit a pull request
The text was updated successfully, but these errors were encountered: