Load DB in integration test #431
Conversation
Good point. Those files don't exist in either integration test; they are integrating the spectro pipeline, not the full end-to-end chain. The minitest.ipynb notebook in two_percent_DESI minitest branch will eventually morph into a more complete integration test, but we're not ready for that yet. Is it at all viable/meaningful to load just the zbest files and skip the rest for now, or do foreign key constraints or similar make that impossible? |
|
In fact there are currently no foreign key constraints imposed on the database tables, so nothing breaks by only loading the zbest files. Only loading the zbest files is a demonstration that it is possible for the pipeline to load them, but not necessarily all that useful beyond that. If that is sufficient, then we're done. If the database loaded by the integration test is supposed to be actually useful, then there is more work to do. |
|
To be clear, I want merging this pull request to define WBS 1852_80 "Finalize Spectro DB Implementation" Due: 25 Aug 2017, being done, as we discussed off line. So if we are still not done with this, I can't merge. Or we need to redefine that item being done. |
|
Focus of this PR has changed to loading |
* master: (91 commits) minor change only fit arc lines in given wavelength range minor updates update default wavelength range Fix typo Break long lines and replace old-style string substitutions with format() adapt pipeline to new psf format, psf merge is now a function of specex in python. there is no more xml intermediate files adapt mean psf to new format adapted pipeline to new specex, now need to change options to use pre-existing psf instead of boot psf Clean up docstring causing sphinx error optionnal improvement of the the pixel flatfield by doing a fast fit of mean spectrum and transmission trying to make travis sphinx happy use sphinx/1.5 Integrate the redrock redshift fitting tool into the pipeline: fix output of resolution fit add --nogain option needed for gain measurement fix bug in format of one debug message several improvements to get the best estimate we can achieve today for the gains write crosstalk values in format of yaml file crosstalk EM1 ...
|
Let's merge this now so that it doesn't dangle and get further out of sync with master, though we'll have to return to this DB loading once we have an obslist with exposure IDs that can be reasonably loaded. |
This PR adds database loading to the old integration test.
This PR is not ready to merge, but is ready for review.
Right now, only zbest files are loaded. I need advice on which exact files to load to populate the target, truth, obslist and fiberassign tables. None of the corresponding files from the two percent datachallenge appear to be generated by the integration test.
The sqlite database is called dailytest.db and appears in the same directory as the dailytest.log file.