Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data extractor disconnects #1653

Merged
3 commits merged into from
May 24, 2022
Merged

Data extractor disconnects #1653

3 commits merged into from
May 24, 2022

Conversation

ghost
Copy link

@ghost ghost commented May 24, 2022

closes #1625

@ghost ghost self-requested a review May 24, 2022 04:18
@codecov
Copy link

codecov bot commented May 24, 2022

Codecov Report

Merging #1653 (9f9ca68) into master (fed55ee) will decrease coverage by 0.01%.
The diff coverage is n/a.

❗ Current head 9f9ca68 differs from pull request most recent head 7428a3a. Consider uploading reports for the commit 7428a3a to get more accurate results

@@            Coverage Diff             @@
##           master    #1653      +/-   ##
==========================================
- Coverage   77.85%   77.84%   -0.02%     
==========================================
  Files         234      232       -2     
  Lines       18161    18122      -39     
==========================================
- Hits        14140    14107      -33     
+ Misses       4021     4015       -6     
Impacted Files Coverage Δ
lib/cosmos/microservices/reducer_microservice.rb 90.78% <0.00%> (-0.61%) ⬇️
config/routes.rb 100.00% <0.00%> (ø)
app/models/completed_script.rb
app/controllers/completed_script_controller.rb

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fed55ee...7428a3a. Read the comment docs.

import * as ActionCable from 'actioncable'
import * as ActionCable from '@rails/actioncable'
//ActionCable.logger.enabled = true
ActionCable.ConnectionMonitor.staleThreshold = 60
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like the default is 6. Do you think this is what did it?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This helped. The multiple files was the other main change.

if (this.bytesReceived > 200000000) {
this.bytesReceived = 0
this.createFile()
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Create a new file every 200MB?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Otherwise the browser uses up tons of RAM. At this amount it still gets up to about 1 GB.

end
false
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're not really using the return value (true/false) anymore. ReducerModel.rm_file(file) everytime. Not sure why that was in an if before but I don't think we ever returned false.

@ghost ghost merged commit 750a34d into master May 24, 2022
@ghost ghost deleted the data_extractor_disconnects branch May 24, 2022 16:50
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Large DataExtractor queries appear to timeout
1 participant