Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Build progress bar only on first update #3626

Merged
merged 4 commits into from
Dec 21, 2024
Merged

Conversation

colin-ho
Copy link
Contributor

@colin-ho colin-ho commented Dec 19, 2024

Theres 2 progress bars for swordfish, a rust based one thats for terminals, and a python one thats for jupyter notebooks (uses tqdm).

This pr makes it so that the tqdm bar is only rendered on first update (the rust based one is already lazy rendered), + increase the refresh rate of both progress bars to 500ms.

native

native.mov

py

py.mov

@github-actions github-actions bot added the chore label Dec 19, 2024
Copy link

codspeed-hq bot commented Dec 19, 2024

CodSpeed Performance Report

Merging #3626 will improve performances by 89.78%

Comparing colin/lazy-progress (4404309) with main (f9a89a7)

Summary

⚡ 1 improvements
✅ 26 untouched benchmarks

Benchmarks breakdown

Benchmark main colin/lazy-progress Change
test_iter_rows_first_row[100 Small Files] 230.1 ms 121.2 ms +89.78%

@colin-ho colin-ho force-pushed the colin/lazy-progress branch from caedb22 to 4ba1200 Compare December 19, 2024 22:39
Copy link

codecov bot commented Dec 19, 2024

Codecov Report

Attention: Patch coverage is 16.66667% with 15 lines in your changes missing coverage. Please review.

Project coverage is 77.90%. Comparing base (063de4d) to head (4404309).
Report is 13 commits behind head on main.

Files with missing lines Patch % Lines
daft/runners/progress_bar.py 6.66% 14 Missing ⚠️
src/daft-local-execution/src/progress_bar.rs 50.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3626      +/-   ##
==========================================
+ Coverage   77.80%   77.90%   +0.09%     
==========================================
  Files         718      720       +2     
  Lines       88250    88726     +476     
==========================================
+ Hits        68666    69122     +456     
- Misses      19584    19604      +20     
Files with missing lines Coverage Δ
src/daft-local-execution/src/lib.rs 95.45% <100.00%> (-0.09%) ⬇️
src/daft-local-execution/src/progress_bar.rs 63.52% <50.00%> (+1.30%) ⬆️
daft/runners/progress_bar.py 57.29% <6.66%> (-6.67%) ⬇️

... and 25 files with indirect coverage changes

@colin-ho colin-ho marked this pull request as ready for review December 20, 2024 22:03
bar_format, initial_message = self.bar_configs[pbar_id]
self.pbars[pbar_id] = self.tqdm_mod(
bar_format=bar_format,
desc=initial_message,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like initial_message isn't really used? It gets added for a brief moment during the first update, but is superceded shortly after.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch

@colin-ho colin-ho merged commit 1c0f780 into main Dec 21, 2024
42 of 43 checks passed
@colin-ho colin-ho deleted the colin/lazy-progress branch December 21, 2024 04:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants