UVES_popler
ESO/VLT UVES post-pipeline echelle reduction
By: Michael T. Murphy

Usage: UVES_popler [OPTIONS] [FITS file or list]

Brief description:


Non-standard requirements:
PGPLOT and CPGPLOT libraries.
CFITSIO library.

Copyright:
Copyright 2003-2015 Michael T. Murphy

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.


Contents
  1. Download and installation
  2. System architectures tested
  3. Update information
  4. Basic Operation
    1. The input FITS file list: ABSOLUTE PATH NAMES!
    2. The files pointed to by file_list.txt: Flat-fielded, bias-corrected, extracted 2D spectra
  5. Options
    General spectrum specifications:
    • -combmeth = 0 : Comb. method, order-scaling (0), order-fitting (1)
    • -disp = TBD : Dispersion in km/s or Angstroms (for lin. disp.)
    • -filetype : File origin: UVES=0, IRAF=1, MAKEE=2, IRAFLSS=3, mixed=-1
    • -zem = 0.00000 : Emission redshift of QSO (if required for cont. fit)
    • -linear = 0 : Linear (1) or log-linear (0) dispersion
    • -thar = 0 : Reduce and combine ThAr calibration spectra
    Initial sigma-clipping of echelle orders (not done when using -thar):
    • -nordclip = 3 : No. most positive pixels removed from clip window
    • -nordsig = 35 : No. pixels in order sig.-clip window
    • -ordsig = 5.00 : Sigma-clipping level for orders [rms in window]
    • -ordsignbr = 3.00 : Sigma-clipping level for neighbouring pixels
    • -ordsigzero = 3.00 : Only clip when mean is this many sigma above zero
    Spectrum/order combination parameters:
    • -clipsig = 3.00 : Sigma-clipping level for spectra combination
    • -lrgerr = 4.00 : Large error rejection ratio for combination
    • -rankspec = 0 : (0) Autoscale; (1) Hi-SNR ord of 1st spec unscaled
    • -scalclip = 3.00 : Sigma-clipping level for inter-order scale-factor
    • -scalerr = 0.20 : Limit for relative error on inter-order scale-factor
    Order continuum fitting method parameters:
    • -contftyp = 2 : Type of fit (1=polynomial, 2=Chebyshev, 3=Legendre)
    • -contord = 4 : Order of fitted continuum (1=const.)
    • -contpctl = 0.10 : Lower percentile to remove for initial fit [%/100]
    • -contsigl = 1.40 : Lower rej. level for order continuum fitting [sigma]
    • -contsigu = 3.00 : Upper rej. level for order continuum fitting [sigma]
    • -contwgt = 500 : Weighting scale in pixels for ends of orders
    Combined spectrum continuum fitting method parameters:
    • -cordlya = 4 : Order of fitted continuum below Lyman-alpha
    • -cordred = 6 : Order of fitted continuum above Lyman-alpha
    • -ftyplya = 2 : Type of fit below Lyman-alpha (1=pol, 2=Che, 3=Leg)
    • -ftypred = 2 : Type of fit above Lyman-alpha (1=pol, 2=Che, 3=Leg)
    • -pctllya = 0.30 : Lower percentile to remove for initial Lya fit [%/100]
    • -pctlred = 0.10 : Lower percentile to remove for initial red fit [%/100]
    • -rsiglyal = 1.20 : Lower rej. level for Lyman-alpha continuum fit [sigma]
    • -rsiglyau = 3.00 : Upper rej. level for Lyman-alpha continuum fit [sigma]
    • -rsigredl = 1.40 : Lower rej. level for cont. fit redwards of Lya [sigma]
    • -rsigredu = 3.00 : Upper rej. level for cont. fit redwards of Lya [sigma]
    • -vclya = 20000.00 : Velocity scale for continuum fit below Lya [km/s]
    • -vcred = 2500.00 : Velocity scale for continuum fit above Lya [km/s]
    • -vlya = 5000.00 : Velocity below Lya where red cont. fit begins [km/s]
    Data file options:
    • -dat = 0 : Write out data file with normalized spectrum
    • -save = 0 : Force UPL and FITS save (1) when quitting program
    Mode options:
  6. User's manual
  7. Output



1. Download and installation:
The best, and now only, way to grab the code is to get it from GitHub. You can grab a zip file from that site or, better, you can install"github" program and then pull the code to your computer, i.e.

prompt> git clone https://github.com/MTMurphy77/UVES_popler.git

After grabbing the code (and unzipping the zip file, if that's what you downloaded), change to the source code directory:

prompt> cd UVES_popler/

You will need to edit the Makefile to reflect your system's architecture, where your PGPLOT, CPGPLOT and CFITSIO libraries are and where you'd like the final executable to be put. For the latter, change the TARGET keyword in the Makefile header. You should then just be able to type

prompt> make depend

prompt> make

and hopefully things will just compile nicely. Let me know if you have problems compiling the code. Afterwards you might like to do a

prompt> make clean

to remove the *.o and temporary *~ files.


2. System architectures tested:
UVES_popler is designed to run on UNIX and Linux systems. So far, it has been tested on a wide variety of linux and UNIX platforms, including linux-based Mac OS versions. UVES_popler should work on any UNIX/Linux installation since the libraries it uses are fairly standard. Let me know if you have problems getting it to work on a specific platform.


3. Update information:
UVES_popler is a new program and it's written by a C-code amateur. I will try to update the available version (as well as this page) when I make changes. Please let me know if you find bugs or you wish UVES_popler to do something differently or something new.

To stay up to date with new versions and changes to UVES_popler, it is best to simply follow it on GitHub.


4. Basic operation:
To execute the program for the first time:

prompt> UVES_popler file_list.txt

You may receive either an error message or several warning messages. These will probably be associated with your input file (file_list.txt) or the FITS files pointed to by that file.
  1. The input FITS file list: ABSOLUTE PATH NAMES!: Getting the input FITS file list correct is crucial to the workings of UVES_popler. This file should be text only and should only contain one file name per line. The crucial thing is that each FITS file should be listed with its full absolute path name. For example, the following file list is NOT valid:

    Q1209p107/fxb_Q1209p107_sci_390_01_b.fits
    Q1209p107/fxb_Q1209p107_sci_390_02_b.fits
    Q1209p107/fxb_Q1209p107_sci_564_01_l.fits
    Q1209p107/fxb_Q1209p107_sci_564_01_u.fits
    Q1209p107/fxb_Q1209p107_sci_564_02_l.fits
    Q1209p107/fxb_Q1209p107_sci_564_02_u.fits

    because each line doesn't begin with a "/". It's also a good idea to have the FITS files (pointed to by file_list.txt) somewhere other than the present working directory (PWD).

  2. The files pointed to by file_list.txt: Flat-fielded, bias-corrected, extracted 2D spectra: The files pointed to by file_list.txt must be FITS files output by the UVES pipeline. Various files are produced by the pipeline but the ones pointed to by file_list.txt must be the flat-fielded, bias-corrected, extracted 2D spectra of the science object. These spectra have been binned in the spatial direction by the optimal extraction procedure but HAVE NOT been redispersed (i.e. re-binned in the dispersion direction).

    The flux files should be labelled with a "fxb_" prefix. The associated error files should have a "err_" prefix and the wavelength calibration files should have a "wpol_" prefix. The "err_" and "wpol_" files must reside in the same directory as their associated "fxb_" files.

    If you used the MIDAS UVES pipeline (now well out of date), the "flx_" and "err_" prefixes are automatically added to the flux and error files. The wavelength calibation information is stored as wavelength calibration polynomial coefficients in tabular form in files named lCWLchipBINXxBINY_2.tbl. Here CWL is the central wavelength of the exposure, chip is "_" for blue settings and "L" or "U" for the bluer and redder of the red UVES chips, BINX and BINY are the spectral and spatial CCD binning factors. For example l390_2x2_2.tbl, l580L1x1_2.tbl and l580U1x1_2.tbl. These files must be converted to FITS format within the pipeline and given the same name as the "fxb_" files but with the "fxb_" replaced by "wpol_". If you used UVES_headsort to sort the raw UVES spectra and write the MIDAS reduction scripts then the naming conventions above are automatically produced via the reduction scripts.

    If you used the modern CPL UVES pipeline using ESOREX routines directly, you will need to have added the "--debug" flag to the "uves_obs_scired" command. This will produce the required "fxb_" flux file and associated "errfxb_" error file. The latter must be renamed to have "err_" as the prefix. If you used UVES_headsort to sort the raw UVES spectra and write the CPL reduction scripts then the naming conventions above are automatically produced via the reduction scripts.

    If you used ESO's Reflex interface to the CPL pipeline, following these instructions (courtesy of Trystyn Berg at ESO) will produce the required files:
    • Open the spectrum extraction actor: i.e. Right-click → Open actor.
    • Configure the uves_obs_scired actor (double-click), and set the option "debug=true" (all lower case).
    • Run the UVES reduction workflow as normal. The relevant "fxb_" and "errfxb_" files appear within the reflex_book_keeping/uves/uves_obs_scired1// directory. The latter must be renamed with the "err_" prefix. The relevant LINE_TABLE file is listed within the sof.dat file in the same directory; it should be remamed to have the "wpol_" prefix.


5. Options:


6. User's manual:
6.2 Initial order cosmic ray cleaning: For each order that is read in, a (nordclip)-pixel window is slid along it and the mean and variance of the pixels in the window (without including the nordclip highest flux pixel) is calculated. Then if the middle pixel is more than (ordsig)-sigma [i.e. sqrt(variance)] above the mean then it is regarded as a cosmic ray and removed. Also, if the pixel immediately to the left of the removed pixel is more than (ordsignbr)-sigma above the mean then it is also removed. The same is true for the pixel to the right of the first removed pixel.


6.5 Status symbol meanings:


6.6 Action IDs:


6.8 Step-by-step approach to a good reduction:
A summary of the six steps I normally take to get a reliable reduction and a UPL file which is easier to follow and understand is as follows:


7. Output: