LoopingCallService sync by default

[etam]

For these services I think it's OK to let them run in threads:

• TaskArchiverService
• ResourceCleanerService

fixes #3660
fixes #3245
resolves #2851
fixes #5083

[mplebanski]

Why do you think it is safe to run TaskArchiver service in a separate thread? By looking at do_maintenance method it is no so clear to me.

[mplebanski]

Same question for ResourceCleanerService. It is using the same dir_manager that is used by the golem apps arbitrarily.

[etam]

Why do you think it is safe to run TaskArchiver service in a separate thread? By looking at do_maintenance method it is no so clear to me.

Ok, I had a closer look at it, and it's not thread safe. But very little work is required to make it so, so I'll do it.

[etam]

Same question for ResourceCleanerService. It is using the same dir_manager that is used by the golem apps arbitrarily.

No, it calls remove_distributed_files and remove_received_files, which are creating their own instances of DirManager and are just removing files from filesystem. There's an issue to make it more safe #2432, but from threading point of view, I see no problem here.

[mplebanski]

Did you have a chance to run it and check performance behavior?

[etam]

Did you have a chance to run it and check performance behavior?

Yes and it seems that it's working good.

[codecov]

Codecov Report

Merging #3661 into develop will increase coverage by 2.23%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           develop    #3661      +/-   ##
===========================================
+ Coverage    87.78%   90.01%   +2.23%     
===========================================
  Files          234      234              
  Lines        22090    22079      -11     
===========================================
+ Hits         19391    19875     +484     
+ Misses        2699     2204     -495

[mfranciszkiewicz]

DoWorkService will now be called every second in the main thread. Please profile it when Golem is under load, since that service used to be a big performance hit.

[mplebanski]

Are we going to merge that or not?

[etam]

I'm working on properly profiling golem. It means I need to learn a bit of this stuff. When I'm done I'll share my results and write a guide for others.

[mplebanski]

Awesome.

[etam]

The same change, but rebased onto b0.22 is in branch LoopingCallService_sync_by_default_b0.22

[jiivan]

IMvHO it this change looks promising. Were there any conclusions from profiling/running under load?

Outdated

[etam]

I didn't got to testing under serious load (like mainnet network), because I was moved to other tasks back then. Now our QA team is testing this (but without profiling), to check if there are any regressions.