Lossy fiber analysis for pixel level flat

[julienguy]

Write software to analyze lossy fiber CCD images. Preliminary work started at DESI June 2017 collaboration meeting. Need to solve simultaneously for average lamp spectrum, fiber transmission.
Difficulties

• there is an apparent chromatic variation of transmission (but slow/smooth variation with wavelength)
• the fiber/wavelength coordinate mapping on CCD image needs to be accurate and it's different from fiber slit.
• more defocusing of the lossy fiber would help

[julienguy]

Work in progress in branch pixelflat.
the pixel flat = data/model

Two methods were developed

• 1. A forward model approach which shows that there are two spectral components. This is interesting for the hardware refinement, but this method cannot be used to compute a flat-field because it leaves too many defects in the flat.
• 2. An empirical model implemented in branch pixelflat

usage: desi_compute_pixel_flatfield [-h] -i [IMAGES [IMAGES ...]] -o OUTFILE
                                    [--sigma SIGMA]                         
                                    [--niter_filter NITER_FILTER]           
                                    [--niter-mask NITER_MASK]               
                                    [--sigma-mask SIGMA_MASK]               
                                    [--nsig-mask NSIG_MASK]                 
                                    [--nsig-smooth-mask NSIG_SMOOTH_MASK]
                                    [--nblocks NBLOCKS]
                                    [--out-median OUT_MEDIAN] [--psf PSF]
                                    [--per-amplifier] [--no-trim]

Computes a pixel level flat field image from a set of preprocessed images
obtained with the flatfield slit. A median image is computed if several
preprocessed images are given. The method consists in iteratively dividing the
median or input image by a smoothed version of the same image, flat(n+1) =
flat(n)/smoothing(flat(n)). The smoothing consists in 1D FFT Gaussian
convolution along the wavelength dispersion axis or the fiber axis
alternatively. A masking of outliers is performed to avoid tails around CCD
defects. The trace orientations is obtained from an input PSF file, and the
orientations are averaged in blocks (the number of blocks is a parameter).
Optionally the modeling can be performed per CCD amplifier to correct for gain
mismatch (possibly due to non-linearities). The method fails in areas where
the illumination pattern varies in both directions at a scale smaller than the
sigma value, but the sigma value is also limited by the maximum size of the
CCD defects to be captured in the flat.

optional arguments:
  -h, --help            show this help message and exit
  -i [IMAGES [IMAGES ...]], --images [IMAGES [IMAGES ...]]
                        path to input preprocessed image fits files, or a
                        single median image (default: None)
  -o OUTFILE, --outfile OUTFILE
                        output flatfield image filename (default: None)
  --sigma SIGMA         gaussian filtering sigma (default: 60)
  --niter_filter NITER_FILTER
                        number of iterations in gaussian filtering (default:
                        2)
  --niter-mask NITER_MASK
                        number of iterations for mask evaluation (default: 3)
  --sigma-mask SIGMA_MASK
                        gaussian smoothing sigma for the mask (default: 10)
  --nsig-mask NSIG_MASK
                        # sigma cut for mask (default: 4.0)
  --nsig-smooth-mask NSIG_SMOOTH_MASK
                        # sigma cut for mask after smoothing (default: 1.5)
  --nblocks NBLOCKS     number of blocks along one axis (total number of
                        blocks is the square) where the trace orientations are
                        averaged (to use in combination with --psf option
                        otherwise ignored) (default: 4)
  --out-median OUT_MEDIAN
                        save output median image (for development) (default:
                        None)
  --psf PSF             use traces in this PSF to orient 1D convolutions
                        (default: None)
  --per-amplifier       solve model per amplifier if gains are uncertain or
                        non-linarities (default: None)
  --no-trim             do not compute flat only in CCD area covered by traces
                        (to use in combination with --psf option otherwise
                        ignored) (default: None)

Example flat (it fails today at the largest wavelength). This work will be resumed with the flatfield slit has been modified.

desi_compute_pixel_flatfield -i preproc-b1-000050{46,47,48,49}.fits --psf psf-b1.fits -o flat-b1.fits --per-amplifier >& desi_compute_pixel_flatfield_b1.log &

[julienguy]

We should leave this ticket open until we have verified the code works on the spectrograph this fall.
Meanwhile, people interested can look at the flats at NERSC in /project/projectdirs/desi/users/jguy/teststand/lossy-fiber