Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reimplement PhaseTDStatistic #3596

Merged

Conversation

titodalcanton
Copy link
Contributor

@titodalcanton titodalcanton commented Jan 26, 2021

Reimplement PhaseTDStatistic based on the code that was in place before #3535, and modify the PyCBC Live test to use it. Some extra changes are needed to fix some errors related to a from_cli() method modifying some of the arguments and breaking further calls to the same function.

Closes #3580. Closes #3584.

I have not tested that the resulting values are actually correct or equivalent to the old code, though, so marking as WIP.

Copy link
Contributor

@spxiwh spxiwh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me. As long as you/others are happy that the new code in stat.py reproduces what was done with the old 2-det coinc code then please merge.

@titodalcanton
Copy link
Contributor Author

titodalcanton commented Jan 28, 2021

Hmm, interesting failure in PyCBC Live, apparently reproducible.

>> [Thu Jan 28 10:00:54 UTC 2021] Checking results
2021-01-28 10:00:55,585 No triggers found in V1
2021-01-28 10:00:55,585 Single Trigger Test Failed
2021-01-28 10:00:55,585 2 coincident trigger(s) detected
2021-01-28 10:00:55,586 Checking trigger output/coinc-1272790100.1669922-zimpiz.xml.gz
2021-01-28 10:00:55,668 IFO SNRs: [9.41339   7.7795067]
2021-01-28 10:00:55,669 Network SNR: 12.211988
2021-01-28 10:00:55,669 IFO End Time: 1272790100.000000
2021-01-28 10:00:55,669 Mass 1: 290.929321
2021-01-28 10:00:55,669 Mass 2: 3.675545
2021-01-28 10:00:55,669 Spin1z: 0.993485
2021-01-28 10:00:55,669 Spin2z: 0.927135
2021-01-28 10:00:55,669 Checking trigger output/coinc-1272790260.0966797-nrxeea.xml.gz
2021-01-28 10:00:55,749 IFO SNRs: [8.146549 6.880121]
2021-01-28 10:00:55,750 Network SNR: 10.663130
2021-01-28 10:00:55,750 IFO End Time: 1272790260.000000
2021-01-28 10:00:55,750 Mass 1: 1.133169
2021-01-28 10:00:55,750 Mass 2: 1.010624
2021-01-28 10:00:55,750 Spin1z: 0.029544
2021-01-28 10:00:55,750 Spin2z: 0.020994
2021-01-28 10:00:55,750 Test Failed
    FAILED!

???

@titodalcanton titodalcanton force-pushed the reimplement_phasetd_stat branch from 7a11b8d to 2fcb6f1 Compare January 29, 2021 10:37
@titodalcanton
Copy link
Contributor Author

For reference, the failing test was due to #3603.

@titodalcanton
Copy link
Contributor Author

Here is a very simple test using the current example. I compared the rankings of the recovered injections between this code version and 503873c (just before the stat refactor).

2020_05_06/H1L1V1-Live-1272790096.069336-8.hdf
  Difference in coa_phase_H1: -1.1920928955078125e-07
  Difference in coa_phase_V1: -2.384185791015625e-07
  Difference in coinc_rank: 0.40607452392578125
  Difference in coinc_rank_squared: 10.332295827902271
  Difference in end_time_H1: 0.0
  Difference in end_time_V1: 0.0
  Difference in ifos: set()
  Difference in snr_H1: 4.76837158203125e-06
  Difference in snr_V1: -7.152557373046875e-06
2020_05_06/H1L1V1-Live-1272790256.069336-8.hdf
  Difference in coa_phase_H1: -1.1920928955078125e-07
  Difference in coa_phase_V1: -3.5762786865234375e-07
  Difference in coinc_rank: 0.42845726013183594
  Difference in coinc_rank_squared: 10.54476247247294
  Difference in end_time_H1: 0.0
  Difference in end_time_V1: 0.0
  Difference in ifos: set()
  Difference in snr_H1: -3.814697265625e-06
  Difference in snr_V1: 9.5367431640625e-07

The difference in coinc ranks seems a bit high (both injections have coinc rank ~12). Part of this difference may be due to the necessary change in the format of the phase-time-amplitude files, however.

@titodalcanton
Copy link
Contributor Author

I have now poked at the old and new phase-time-amplitude files for some time (see plots here) and I convinced myself that the ~100x difference in expected signal rate comes from the phase-time-amplitude files themselves and not from this code change. I might look into this further, but I would like to merge this so we can have a working O3-like configuration of PyCBC Live.

@titodalcanton titodalcanton changed the title [WIP] Reimplement PhaseTDStatistic Reimplement PhaseTDStatistic Feb 3, 2021
@tdent
Copy link
Contributor

tdent commented Feb 3, 2021

@titodalcanton have you had any feedback from @ahnitz on the change in PTA files and resulting change in statistic?

@titodalcanton
Copy link
Contributor Author

@ahnitz and I discussed this and agreed that the rate is expected to have differences due to the PTA file format change.

@titodalcanton titodalcanton merged commit c43fcbb into gwastro:master Feb 5, 2021
@titodalcanton titodalcanton deleted the reimplement_phasetd_stat branch February 11, 2022 14:55
OliverEdy pushed a commit to OliverEdy/pycbc that referenced this pull request Apr 3, 2023
* Reimplement PhaseTDStatistic

* Fix error in PyCBC Live due to args.statistic_files being modified by from_cli()

* Switch PyCBC Live test to use the PTA statistic again

* A couple fixes from Codeclimate

* Add reference for coinc stat

* Add safety check for coinc_lim_for_thresh()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PyCBC Live: re-enable the O3 coinc ranking PyCBC Live test: generate PTA files and use a PTA-based ranking
3 participants