Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unique dat and log file names #59

Merged
merged 3 commits into from
May 8, 2022

Conversation

lrampa
Copy link

@lrampa lrampa commented May 8, 2022

Hi, I would like to execute the same DAG in multiple instances in parallel using containers, and gather the dat and log files from all runs in one common place (same as it is now under ~/.dagu).
This pull request adds request id to all file names to make them unique, and milliseconds for more precise tracking.

I am not sure about the change in Agent.Run() method though. It is now somewhat ugly (calling setupRequestId() before init()). If you see better way how to handle this point, I would be glad to change it.

Thanks, Lukas

Lukas Rampa added 3 commits May 7, 2022 03:06
Request Id is used in log and dat file names to prevent files overwriting
when executing parallel DAGs on shared filesystem.
@codecov-commenter
Copy link

Codecov Report

Merging #59 (9ab8640) into main (b5da55d) will decrease coverage by 0.08%.
The diff coverage is 84.21%.

@@            Coverage Diff             @@
##             main      #59      +/-   ##
==========================================
- Coverage   84.21%   84.13%   -0.09%     
==========================================
  Files          37       37              
  Lines        2585     2590       +5     
==========================================
+ Hits         2177     2179       +2     
- Misses        295      297       +2     
- Partials      113      114       +1     
Impacted Files Coverage Δ
internal/agent/agent.go 80.25% <62.50%> (-0.95%) ⬇️
internal/database/database.go 88.44% <100.00%> (ø)
internal/scheduler/node.go 83.60% <100.00%> (+0.13%) ⬆️
internal/scheduler/scheduler.go 87.58% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b5da55d...9ab8640. Read the comment docs.

@yottahmd
Copy link
Collaborator

yottahmd commented May 8, 2022

Hi, it looks fantastic. Thank you so much, @lrampa !

@yottahmd yottahmd merged commit 2079274 into dagu-org:main May 8, 2022
yottahmd added a commit that referenced this pull request May 9, 2022
yottahmd added a commit that referenced this pull request May 9, 2022
@yottahmd
Copy link
Collaborator

yottahmd commented May 9, 2022

Hi @lrampa, could you please give it a star? so that github can show you in Contributors page. Thanks!

@lrampa
Copy link
Author

lrampa commented May 9, 2022

Great, thanks for accepting.

@lrampa lrampa deleted the feature/unique_file_names branch May 9, 2022 08:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants