Required science analysis programs
----------------------------------
(Updated 3-Oct-95, CDP) Including new requirements from Richard.
The following is a general list of requirements for
software to enable CDS data users to analyse data easily
and efficiently.
The specific originator of the request is identified so
you can get back to them for them to explain what it is
they really had in mind. If you have any further
suggestions, specific ideas on solutions (eg - would be
satisfied by an interface to existing routine xxx), or
would like to volunteer to work on any of the items then
please contact either:
Dave Pike (cdp@astro1.bnsc.rl.ac.uk) or
Bill Thompson (thompson@orpheus.nascom.nasa.gov).
----------
1) (Art) Select data from multiple FITS files. Combine
monochromatic images into time series and display as a
movie.
Suggestion is to interface CDS data cube (images v time)
to Yohkoh xstepper routine.
OSLO....
We do not see any problems in doing readouts of monochromatic
images from multiple FITS files, feeding this to either a
"Yohkoh stepper-like" routine or a Xanimate-like routine.
Oslo may undertake this task, but some questions need to be
discussed. These include possible corrections for pointing
drift and/or solar rotation in order for time series to display
the same area on the sun over extended periods of time.
CDP...
There is a further major problem in 'simply' extracting a
monochromatic (or nearly so) image from a data cube (or series of FITS
files) and that is that the data must be wavelength calibrated first.
Since the wavelength calibration is a function of the mirror, slit
positions etc.(extent TBD) depending on the wavelength range
extracted it may not make sense to even attempt an extraction without
the calibration.
I suggest there be a tag in the QLDS to flag whether wavelength
calibration has been performed or not.
----------
2) (Art) Display many images of same event/area and be
able to select which are selected for inclusion in movie
OSLO...
We feel this task to be of the same type as 1) and would
consider doing it.
-----------
3) (Art) Apply calibration to all or selected data
VDS routines are available.
MSSL are working on routines to apply the various corrections
needed for GIS data.
----------
4) (Art) Remove spectral and spatial MTF
----------
5) (CDP) Calculate line intensities from multiple
line/multiple exposures by summation/profile fitting
etc.
Don has expressed an interest in this. Dominic says he has
some routines which are available form the Yohkoh tree.
OSLO...
There seem certainly to be enough people taking an active interest
in this. We only like to raise some question marks at methods
gaining information by profile fitting. (See the CDS report by
Brynildsen).
A narrow line profile may have only one Gaussian element and
fitting is then OK. Where the profile is broad there may be a
number of profiles involved, but we would not know, because the
CDS spectral profile is so broad relative to the widths and
spacing of multiple components. We base this on experience with
HRTS-data. SUMER may be in a better position, but even then it
will be difficult to derive any reliable information in the case
of fitting more than one profile. Thus, fits like this might lead
to physically incorrect information.
See
----------
6) (Andrzej/Art) DEM calculation
Alessandro Lanzafame (working with Hugh Summers in
Glasgow) has resurrected a Fortran DEM code and has an
IDL interface. Uses atomic data produced by ADAS. In
the future Andrzej's own DEM code will also be
integrated into this system.
OSLO...
This we would think is well in hand by the efforts of Lanzafame,
Summers and Andrzej a.o. We imagine, however, that there may be
other procedures one might include and would draw the attention
Phil Judge (HAO) who has expressed an interest in this. He has
developed the HAO-DIAPER code in IDL which provides DEM from
observed data. Paal Brekke will for the time being serve as an
interface between the CDS software and the HAO-DIAPER.
P Judge...
HAOS-DIAPER and its application to data from CDS and SUMER.
Basically the data and procedures in HAOS DIAPER provide the
currently documented functions of the ADAS package, plus
extras. Being a completely independent project this can serve
to check the ADAS data and its functions. All data are stored
in IDL variables so that manipulation and use is very simple.
At this time I give one example that may be of interest. An
interface between atomic emissivity calculations and observed
data has been completed (it uses the same IDL database software
used by CDS), which allows one to derive temperatures,
densities and (later) differential emission measures from
observed data which can be a scalar line ratio, a vector (data
along the NIS slit), or a 2D image array. Thus one can produce
a "density map" or "temperature map" from the ratio of two
intensity maps and two wavelengths specifying the particular
emission lines studied. This might be of interest for quick
look software.
----------
7) (General) Calculate synthetic spectrum for CDS/Sumer
range
Glasgow group will also provide this - again based on
ADAS data.
The Dere code has had a front end put on it. Try the procedure
DMM_SS (Dere, Mason, Monsignori-Fossi Spectral Synthesis)
Don has also expressed an interest in this.
----------
8) (Art) Plot line profiles of selected lines and
measure line shifts to get v(x,y,t,T)
OSLO..
This seems an extension of 5). If we interpret the request as
obtaining the shifts from a fit to a single profile component or
from a moment calculation it seems OK. The point is then to sort
out and flag the areas where a single component approximation is
insufficient. We would like to discuss this further with others,
and may contribute routines if convenient.
----------
9) (Art) Cross correlate spectra to get shifts
OSLO...
We may also mention that Phil will spend time in Oslo as a
visiting scientist for much of 1996. We wonder if we may
contribute good routines for finding shifts, but are not certain
that we have interpreted the requirement correctly. Olav will ask
Art to clarify.
----------
10) (Art) Retrieve atomic data for specific lines for
use in non-LTE analysis
Probably catered for by ADAS - I will get details later
Don has expressed an interest in this.
----------
11) (Andrzej) Get Solar (x,y) by clicking on any CDS
image
Display CDS image and use structure information to get
coordinates correct.
See available SERTS routines first. Need routine to extract a
wavelength slice (see eg gt_image, cds_image and functionality
within the QL) and plot it with axes (cf plot_image).
OSLO...
Oslo may look into this. There already exists a routine to
extract a wavelength slice and it is easy (see comments to (1) - ed.)
to then build up an image representing any bandwidth in wavelength.
The proposed functionality of GT_IMAGE will return solar X/Y
coordinates in 2-D arrays.
We imagine that this task also will include number 18 (Helen).
----------
12) (Andrzej) Mark any polygon on a CDS data image and
return spectrum which is the sum of all spectra internal
to the polygon.
See available SERTS routines first.
Done - see POLY_SPEC
----------
13) (Andrzej) Overlay images produced by
SOHO/Yohkoh/groundbased.
Use existing routines from image_tool? See available
SERTS routines first.
----------
14) (Bill) Create a dummy FITS file by combining
vds/gis_dummy and a sci_details database entry.
Proposal is now to create a telemetry file so dummy data
can be run through the complete conversion process.
Initial work done. (see SIM_RASTER)
----------
15) (Dave) Make gt_ functions work as an interface to
data structure.
Routines updated to cater for most common data formats - awaiting
update to QL data structure to be able to handle other cases.
GSFC... Don Luttermoser has been checking these out and found
some inconsistencies
CDP... Any future updates of the existing gt_xxx functions should
delete the 'calibration' option which only complicates the
code. I cannot see that we will ever need to use them
on the data files created during the laboratory calibration
exercise.
OSLO...
I (SVH) think almost all the routines getting out data from
a file/data structure should be packaged into the GT_XXXX
scheme. This means a discontinuation of e.g.,
MAXDATA -> GT_MAXDATA
MINDATA -> GT_MINDATA
NEXPOSURES -> GT_NUMEXP
POINTSLICE -> GT_SPECTRUM
WAVESIZE -> GT_WINSIZE
DETDATA -> GT_DATA (?)
WINDOWNO -> GT_WNUM
WMAX -> GT_MAXDATA (with specifed window number)
We refer to Appendix 1 for further detail.
----------
16) (All) Ensure that all capabilities of QL program
are available as separate command-line programs.
Stein Vidar/Paal can possibly start on this towards end of July.
OSLO...
SVH: After making the GT_XXX routines this is probably not a
difficult job. One would like some more specific requests,
however. Some QL abilities are intrinsically "Widgety",
meaning that a command line interface would simply consist
of manually invoking the separate QL applications
(which is currently possible!).
We're definitely a bit behind the stated schedule for this, btw.
-----------
17) (Helen) Display spectrum at given solar (x,y)
location. Display synthetic spectrum alongside and
allow line identification via cursor using suitable
solar line list.
-----------
18) (Helen) Create spatial image for any wavelength or
range of wavelengths. Try gt_image and see note (11).
-----------
19) (Helen) Calculate temperature/density from selected
line ratios
OSLO...
Phil Judge/Paal Brekke expresses an interest in this. Plan is
to interface the HAO-DIAPER code with co-spatial and
co-temporal CDS spectrograms) in two or more lines to produce
temperatures and densities. These may be realized in map-form
if the spectra form raster images, or be time sequences of a
series of points (f.ex. along the slit in NIS) or time maps
even, depending[ending on the program that has produced the
data.
Regarding these two points, 6) and 19) we think that our
contribution is in the form of an addition to what else is
being done and that it should not replace any other activities.
-----------
20) (Helen) Subtract or add raster data, due account
being taken of any spatial differences.
-----------
21) (Helen) easy multi-Gaussian fit to selected regions
in a spectrum. Fix/float parameters as needed.
-----------
22) (OSLO) We would again like to state the need for a reliable
"test suite", of structured data to use for the verification of
gt_ the procedures.
-----------
23) (RAH) The following is a list derived from a survey of the
requirements of the studies defined in the BB. They may duplicate some
things already on the list but are included here for completeness.
CDS Data Display Software Requirements
Richard A. Harrison, October 2 1995
Version 1
Spectra - GIS and NI
----------------------
** Many of these are noe catered for by the gt_xxx functions **
1. Plot spectrum for specified wavelength (pixel) range or complete
detector range, for any raster exposure or group of exposures, and for
any raster location (image element) or group of locations.
2. Fit a theoretical spectrum to a displayed spectrum.
3. Overlay more than one spectrum for the same wavelength range.
4. Run a `movie' of the evolution of a spectrum for a pre-defined group
of raster locations (image elements).
5. Subtract or take ratio of two spectra (perhaps including a
normalisation factor) to highlight differences.
6. Produce an image of the VDS detector intensity map for any given
spectral and spatial region, for any given exposure or group of
exposures, and for any raster location or group of locations.
7. Run a `movie' of no. 6.
8. Subtract or take ratio of any two VDS detector maps.
9. Fit a line, subtract background and identify the integrated
intensity of the line.
Images - GIS and NIS
--------------------
10. Produce an intensity map for any selected window for a raster or
any selected area of the raster.
11. As with 10, but for a selected sub-window (i.e. if the wavelength
band required is less than the returned window).
12. Produce a composite display of intensity rasters (e.g. 2x2, 2x4,
4x4 - not overlayed).
13. Produce a intensity ratio map for one raster, for two selected
wavelength bands as defined by 10 or 11.
14. Plot a Dopplergram of a selected map (as in 10 or 11).
15. Plot an image composed of a number of CDS rasters (e.g. with the 9
image stack of the N-S Synoptic scan, or the full Sun).
16. Overlay GIS and NIS images of the same lines (for co-alignment).
17. Difference normalised GIS and NIS images as in 16.
18. Produce `movies' of 10, 11, 13, 14, 15.
Others
------
19. Plot the time intensity curve of any given image location or group
of location, for any given wavelength band, over any given number of
exposures.
20. Do the same for the ratio of two intensities as 19.
21. Plot the DEM of any group of pixels for a given set of lines, for
any group of exposures.
22. Plot DEM maps, for a series of rasters, for a range of
temperatures.
23. Plot temperature and density maps, based on ratios from two bands
for a given raster area over a specified time.
24. Produce movies of 22 and 23.
25. Plot as with 15, but for temperatures, densities or EM.
26. Overlay CDS and SUMER images of the same wavelength bands.
==============================================================================
Appendix 1
Additional comments from Stein Vidar
I've started working on the GT_xxxx routines. As I'm moving along
designing and implementing them, I'll start cleaning up the QL
software to make full use of the GT_xxxx routines. I'll hopefully
also get some insights into how to provide more command line
interfaces/applications as well.
I'm going to concentrate on the interface to the "data cubes". Some
of the existing data extraction routines will be discontinued/renamed
to fit the GT_xxxx scheme.
In order to test the QL software (and the GT_xxxx) routines, we'll
need a "test suite" of fits files. If they already exist, that's
fine. If not, could someone (Bill?) take on the challenge of creating
these, matching the list of possible formats. We have (I think...) the
necessary setup here in Oslo in order to do it, but I'd prefer having
someone else beat me to it :-)
I see that Don Luttermoser has found some inconsistencies in some of
the existing gt_xxxx routines. I'll look into some of these (if I get
the time) while he is on vacation.
PRELIMINARY description/suggestion of
GT_xxxx functionality.
I would strongly recommend having a one-to-one relationship between
the function name and the dimensionality of the returned data. This
includes the *identity* or the physical meaning of the dimensions,
hiding most of the differences between the GIS/NIS detectors.
When one of the dimensions is wavelength, there should be a choice
between deselecting into a full detector (single wave band only, I
think) or using a packed format.
Furthermore, the routines should supply (through output keyword
parameters) all the information on the physical coordinates of the
retrieved data such as wavelength, solar x/y etc. The values are
returned in arrays with the same dimension as the data, so that e.g.,
possible non-orthogonality may be accommodated, as well as facilitating
vector operations on the data.
Based on these assumptions, I've come up with the following
tentative description of some GET (gt_xxx) procedures:
GT_PRODUCTS,QLDS
Lists the possible GT_xxx data products for the given
data structure
GT_SOLARX ; Return the (correct) solar coordinates for
GT_SOLARY ; specified points in the data cube.
GT_DELTAT ;
; Input keywords (not all will be applicable at any one time).
; The points in the data cube must be *uniquely* determined
; by the input values, so that the returned array dimensionality
; is always REFORM(FINDGEN(N_WAVEIX, N_XIX, N_YIX, N_TIX),
; where N_WIX is N_ELEMENTS(WIX)
XIX=N X index
YIX=N Y index
WAVEIX=N Wavelength index
BAND=N Wavelength band
TIX=N Time index
GT_MAXDATA / GT_MINDATA
WINDOW = N/'LABEL'
; If window is not specified it takes the global max/min.
GT_IMAGE
; Input keywords (not all will be applicable at the same time).
LAMBDA = N (or [lambda1,lambda2] --> SUM/AVERAGE)
WAVEIX = N (or [N1,N2,...] --> SUM/AVERAGE)
WINDOW = N --> WAVEIX RELATIVE TO WINDOW LIMITS
DETECTOR = N --> WAVEIX ABSOLUTE DETECTOR ADDRESS)
; Output
SOLARX = OUTPUT, SAME DIMENSIONS AS THE IMAGE
SOLARY = OUTPUT, SAME DIMENSIONS AS THE IMAGE
DEL_TIME = OUTPUT ---------------------------
GT_SPECTRUM
; Returns 1-D spectrum
; Input keywords (not all will be applicable at any one time).
XIX = N (X index
YIX = N (Y index
TIX = N (Time index)
BAND = N (Wavelength band)
WINDOW = N/'WINDOW_NAME'
/NODESELECT
; Output:
LAMBDA = OUTPUT (SAME DIMENSIONS)
SOLARX/Y (IF NECESSARY -- SHOULDN'T BE?)
GT_SCANX
GT_SCANY
; Returns 2D scan, either (LAMBDA,SOLAR_X) OR (LAMBDA,SOLAR_Y)
; Input keywords (not all will be applicable at any one time).
XIX OR YIX = N
WINDOW
BAND
/NODESELECT