Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Propagate tags from fw_spec to metadata #345

Merged
merged 17 commits into from
Jun 17, 2023
Merged

Conversation

sivonxay
Copy link
Contributor

Picking up #320.

I think the changed originally proposed would add alot of redundant information into the document/db.

@codecov
Copy link

codecov bot commented Jun 12, 2023

Codecov Report

Merging #345 (75b92cd) into main (26608c5) will decrease coverage by 0.14%.
The diff coverage is 100.00%.

Additional details and impacted files
@@             Coverage Diff             @@
##              main     #345      +/-   ##
===========================================
- Coverage   100.00%   99.86%   -0.14%     
===========================================
  Files           19       19              
  Lines         1430     1439       +9     
  Branches       361      364       +3     
===========================================
+ Hits          1430     1437       +7     
- Misses           0        1       +1     
- Partials         0        1       +1     
Impacted Files Coverage Δ
src/jobflow/managers/fireworks.py 100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

@sivonxay
Copy link
Contributor Author

After speaking to @samblau, it seems that the functionality desired is to propagate the tags from the fw_spec into the metadata stored in the document.

While the documentation recommends to use Job.update_metadata({"tags": ["test"]}) function, people who are starting to use jobflow who have previously used fireworks+atomate are used to using the atomate.common.powerups.add_tags() function to add the tags. Because the tags aren't added to the metadata, they aren't propagated into the documents by jobflow.

@sivonxay sivonxay changed the title Check to see if tags have propagated to the doc [WIP] Propagate tags from fw_spec to metadata Jun 12, 2023
@sivonxay sivonxay marked this pull request as ready for review June 13, 2023 00:20
@sivonxay sivonxay changed the title [WIP] Propagate tags from fw_spec to metadata Propagate tags from fw_spec to metadata Jun 13, 2023
Copy link
Member

@utf utf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. I just had one minor comment.

# Add the metadata from the fw_spec
fw_tags = fw_spec.get("tags", None)
if fw_tags is not None:
job.metadata.update({"tags": fw_tags})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should also check if tags already exist and if they are a list, and in that case append the tags?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright. Added.

One small issue I encountered here was that tags were getting duplicated if the job metadata had the "tags" key in it. Because of this line: spec.update(job.metadata) # add metadata to spec.

To account for this, I just removed all duplicates from job.metadata["tags"] assuming that duplicates will never be desired.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good catch. Thanks!

@utf utf merged commit cee2007 into materialsproject:main Jun 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants