feat(source_removal,beam): updates to holography-based source removal [WIP]

[ssiegelx]

No description provided.

[ssiegelx]

@tristpinsm This PR contains most of the changes that I made to enable proper handling of polarisation in the source subtraction. It also contains a few new tasks that I found necessary to apply to the data prior to stacking over observations.

I have a bunch of configuration files that I can share to create stacked holography templates and then fit them to the data. I can also point you to the already existing templates that I have on disk.

[tristpinsm]

ok, thanks for posting this here! I will try and get the pipeline running myself, so if you could point me to your config files / templates that would be great.

[ssiegelx]

@tristpinsm The config files can be found here: cedar:/scratch/ssiegel/share/beam_config

Let me know if you have any access issues.

Unfortunately it appears that the templates were deleted from my scratch space. I'm going to go ahead and re-generate them now, but they probably won't be ready until after the workshop.

There is a version of the config files that attempts to subtract a single observation. I think you could use this to get things running if you want to work on this problem now.

[tristpinsm]

I also just remembered that I had a similar need for a container that is an array of containers in my gated holography pipeline, so I created a generic ContainerArray class (it's in draco, branch tpm/gating_proc). It's a little clunky, but maybe it could work here too.

[tristpinsm]

@tristpinsm The config files can be found here: cedar:/scratch/ssiegel/share/beam_config

Let me know if you have any access issues.

Unfortunately it appears that the templates were deleted from my scratch space. I'm going to go ahead and re-generate them now, but they probably won't be ready until after the workshop.

There is a version of the config files that attempts to subtract a single observation. I think you could use this to get things running if you want to work on this problem now.

thanks! I'll take a look

[tristpinsm]

I've been looking through the various config files, and here's what I think I could try running:

• Take a single processed holography transit of CygA and apply flags, fit a model to derive a gain and apply it.
• For a given sidereal stack create a stacked beam from this transit.
• Use the stacked beam to solve for a beam model on that sidereal stream and subtract it.
• Generate ringmaps and a delay spectrum, maybe with a delay filter applied as well.

However when I started with the first step, I wasn't able to submit a job because the task ch_pipeline.analysis.flagging.ApplyInputFlag is missing. Could it have slipped through the cracks, @ssiegelx ?

[ssiegelx]

That sounds reasonable. I included ApplyInputFlags in PR #114. I think we can probably merge that if you are happy with it.

[tristpinsm]

That sounds reasonable. I included ApplyInputFlags in PR #114. I think we can probably merge that if you are happy with it.

Oh I see. Yeah I don't have any more comments for that PR.