Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write co-registration settings to JSON param file #60

Merged
merged 12 commits into from
Mar 24, 2022
Merged

Conversation

alex-l-kong
Copy link
Contributor

What is the purpose of this PR?

Closes #54. Currently, after running update_coregistration_params.ipynb, the new conversion factors are appended directly to settings.py. This causes a lot of clutter and causes settings.py to diverge from main. We should write the params to untracked temp .json file instead.

How did you implement your changes

The following changes have been added:

  • write_coreg_params has been nuked: instead we save to a file called stage_optical_coreg_params.json in toffy directly in update_coregistration_params.ipynb.
  • convert_stage_to_optical now includes a a default argument stage_optical_coreg_path set to stage_optical_coreg_params.json. If this path exists, then we read the coregistration conversion factors directly from the created .json file. Otherwise, we use the default factors in settings.py.

@alex-l-kong alex-l-kong self-assigned this Mar 19, 2022
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

Copy link
Member

@ngreenwald ngreenwald left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, this is a good start.

I think we should standardize the coreg file naming, and not give the user the option to modify it. In addition, lets completely remove the coreg setting defaults from settings.py. If there's no coreg file detected, the fucntions that rely on one should give the user an error message and direct them to run the notebook. The defaults are not going to transfer across instruments, so I think it makes more sense to force the user to run it once on each instrument.

If we have a structured name scheme (i.e. coreg_params_yymmdd), then we can search for the files in the expected places.

The alternative is to append successive coreg updates to the same file (coreg_params.json) with some text in the JSON about when they were added. Whichever you prefer.

@alex-l-kong
Copy link
Contributor Author

@ngreenwald I think it's easier to write successive coreg updates to the same file. We'll store each different co-registration parameter set in list format such as:

{
    'coreg_params': [
        {
            'STAGE_TO_OPTICAL_X_MULTIPLIER': ...,
            'STAGE_TO_OPTICAL_X_OFFSET': ...,
            ...
            'date': '220322'
        },
        {
            ...
            'date': '220323'
        }
    ]
}

In this way, we have one JSON file (as opposed to multiple that would create more clutter in toffy and be harder to keep track of) and simply take the most recent set of params created (the last element in 'coreg_params').

This will allow us to remove the STAGE_TO_OPTICAL_... params from settings.py. However, we'll still need to keep the MICRON_TO_STAGE_... constants around somewhere. I think it's easiest to leave these in settings.py...alternatively, we could put these in a separate .json file as well. I'm open to other ideas here.

@alex-l-kong
Copy link
Contributor Author

Per our meeting, the following have been updated:

  1. settings.py no longer contains STAGE_TO_OPTICAL_ constants, however we'll leave the MICRON_TO_STAGE_ constants there
  2. The STAGE_TO_OPTICAL_ constants generated by update_coregistration_params.ipynb will now be saved to toffy/coreg_params.json. It will contain a key called coreg_params which defines a list of every set of conversion factors generated by the user. Each set of co-registraion params will also contain a key called 'date' which will indicate the timestamp they were generated. Only the most recent (the last one appended to the coreg_params list) is used.
  3. If the user has not run update_coregistration_params.ipynb before autolabel_tma_cores.ipynb, an error message will direct them to run the co-registration script first.

@alex-l-kong alex-l-kong requested a review from ngreenwald March 22, 2022 21:45
Copy link
Member

@ngreenwald ngreenwald left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a typo

toffy/tiling_utils_test.py Outdated Show resolved Hide resolved
@alex-l-kong alex-l-kong requested a review from ngreenwald March 23, 2022 20:00
@ngreenwald ngreenwald merged commit f63b9ee into main Mar 24, 2022
@ngreenwald ngreenwald deleted the coreg_save branch March 24, 2022 17:07
@camisowers camisowers added the enhancement New feature or request label Sep 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Save new co-registration params to temp file rather than settings.py
3 participants