
\documentclass[11pt]{article}

\usepackage[dvips]{graphicx}      %% FOR INCLUDING GRAPHICS
\usepackage{natbib}

%% ----------------------------------------------------------------------
%% Add specials here
%% ----------------------------------------------------------------------

\def\ion#1#2{#1\,{\sc #2}\relax}  %% 

%% ----------------------------------------------------------------------
%% Modify: 
%%   Note number, version number and date
%%   Title
%%   Author and affiliation 
%% ----------------------------------------------------------------------

\setlength{\oddsidemargin}{0in}         %  1 inch (0 + 1) from left.
\setlength{\topmargin}{-0.5in}          %  1 inch (1.5 - 0.5) from top.
\setlength{\textwidth}{6.5in}           %  1 inch (8.5 - 1 - 6.5) from right.
\setlength{\textheight}{9in}            %  1 inch (11 - 1 - 9) from bottom.
\setlength{\parindent}{0pt}

\addtolength{\parskip}{0.5\baselineskip}

%  The environment centerpage centers text vertically on the page.  This is
%  useful within a titlepage environment.

\newenvironment{centerpage}{\mbox{} \protect\vspace*{\fill}}{
        \protect\vspace*{\fill} \mbox{} \protect\\ \mbox{}}

\newcommand{\lam}{$\lambda$}
\newcommand{\arcsec}{$^{\prime\prime}$}
\begin{document}                
%\bibliographystyle{plain}               % Define bibliography style.

\begin{titlepage}
\hrule
\noindent\textbf{EUV IMAGING SPECTROMETER}\\
\vspace{-0.7\baselineskip}\\
{\noindent\Huge\bf Hinode}
\vspace{2mm}
\hrule
\vspace{3mm}
\centerline{\bf EIS SOFTWARE NOTE No. 19}
\vspace{3mm}
\hrule
\noindent Version 2.2 \hfill 6 December 2019
\vspace{2mm}
\hrule

\begin{centerpage}

\begin{center}
{\Large\bf How to access and use the EIS databases}\\
\mbox{}\\
\mbox{}\\
Peter Young\\
NASA Goddard Space Flight Center\\
Greenbelt, MD 20771\\
U.S.A.\\
\end{center}

\end{centerpage}

\end{titlepage}

%% ----------------------------------------------------------------------
%% The document!
%% ----------------------------------------------------------------------

\section{Overview}

The EIS project makes use of the same IDL database structures and
software that were used for the SOHO/CDS mission. These in turn were
developed by Don Lindler for the HST/GHRS and IUE astronomy missions,
and are referred to as the ``UIT database system''.

An important reference document on how the databases work and how they
can be accessed is available as CDS Software Note \#10, available from
the CDS homepage (http://solar.bnsc.rl.ac.uk) and from the soho/cds
branch of SSWDB. The reader is referred to this document for more
details. The present document contains examples on how information
about EIS studies can be obtained from the EIS databases.

A key feature of the UIT database system is that large amounts of
information can be accessed very quickly, making it relatively easy to
do complex searches on different parameters. 



\section{Summary of the EIS databases}

There are several different EIS databases but this document will
consider those databases that are most relevant to the general EIS
user. 

\subsection{The as-run database}\label{sect.asrun}

The two main databases are those that list the properties of the
studies and rasters that were run by the instrument -- the ``as-run''
database. (Note: there is an ``as-planned'' database for those studies
that were scheduled to be run by the EIS Chief Observer, but not all
of these studies will have actually been run.)

The as-run database actually consists of two distinct databases,
called 
\verb|eis_main| and \verb|eis_experiment|. They are located in the
directory

\begin{verbatim}
$SSW/hinode/eis/database/catalog
\end{verbatim}

\verb|eis_main| contains a list of all of the timeline entries, while
\verb|eis_experiment| contains a list of all the individual
rasters. To illustrate the difference, consider how an EIS operation
is planned.  The EIS Chief Observer (CO) places a study on the EIS
planning. The study can consist of multiple rasters, although usually
it contains only one raster. The CO can set the study to have multiple
raster repeats. Consider the following example:

\begin{tabular}{lll}
Study  & Raster 1 & 1 run\\
       & Raster 2 & 10 runs\\
\end{tabular}

where the study consists of two rasters, the first raster is run once,
while the second raster is run 10 times. The study has a unique
timeline ID (\verb|tl_id|) associated with it and will have a single
entry in the \verb|eis_main| database. 

In the \verb|eis_experiment| database, there will be 11 entries that
each have the same \verb|tl_id| value as the study in
\verb|eis_main|. These correspond to the 11 rasters of the study. Note
that each of the 11 rasters will end up as a separate FITS file.

\subsection{The raster and line-list databases}

The two as-run databases do not contain all the information about the
rasters that have been run, and so it may be necessary to extract
information from the raster and line-list databases. The former
contains information such as the raster step size, field-of-view and
exposure time, while the latter contains the list of emission lines
observed by the raster. 

The raster (\verb|eis_raster|) and line-list (\verb|eis_linelist|)
databases are stored in: 
%
\begin{verbatim}
$SSW/hinode/eis/database/planning_db/technical_db
\end{verbatim}


\subsection{The as-planned databases}\label{sect.asplanned}

There are four databases related to the timeline, ``as-planned''
database, and they are stored in 
%
\begin{verbatim}
$SSW/hinode/eis/database/planning_db/timeline_db
\end{verbatim}
%
Basically these store the information for the plans that the EIS COs
prepare. Note that not all of the planned studies will actually run.
For example, a study may be aborted if EIS hits its telemetry limit.

Information about the rasters is stored in
\verb|eis_science2_db|. One difference with the \verb|eis_experiment|
database is that repeats of a raster will not have an entry. Consider
the example from Sect.~\ref{sect.asrun}. In the as-planned database,
this \verb|tl_id| will only have two entries: one each for the two
different rasters. Although the second raster was run 10 times, only
the first of the rasters has an entry in the as-planned database.

\section{Examples}

\subsection{Searching by time}

The following gives an example of how the EIS catalog files can be
searched. Suppose we want to find the raster that was running at
22:40~UT on 2007 January 20. The sequence of commands is as follows:

\begin{verbatim}
dbname=getenv('SSW')+'/hinode/eis/database/catalog/eis_experiment'
dbopen,dbname
t_obs='22:40 20-jan-2007'
t_tai=anytim2tai(t_obs)
t_tai_str=trim(t_tai,'(f15.3)')
query='date_obs < '+t_tai_str+', date_end > '+t_tai_str
list=dbfind(query)
dbext,list,'filename,tl_id',filename,tl_id
dbclose
print,filename
\end{verbatim}

which will return the filename \verb|eis_l0_20070120_223207.fits.gz|.

The reader should note the key parts of this query:

\begin{enumerate}
\item Open the database (dbopen).
\item Create a query string.
\item Perform the query (dbfind).
\item Extract information from the database (dbext).
\item Close the database (dbclose).
\end{enumerate}

When determining what fields to extract from the database, the user
should do:

\begin{verbatim}
dbhelp
\end{verbatim}

(when the database is open) and a list of all of the fields will be
displayed. The user can choose from any of these when making a query
or extracting information.


\subsection{Extracting study information}

The eis\_experiment database only contains information from the raster
database. Suppose that we want to extract some of the meta-data from
the study database that is associated with the raster found in the
previous example.

The key database item that maps the raster database to the study
database is the timeline ID number (TL\_ID). In the previous example
we extracted the timeline ID, which we find to be 697. We now open the
study database to extract information for this timeline ID.

\begin{verbatim}
dbname=getenv('SSW')+'/hinode/eis/database/catalog/eis_main'
dbopen,dbname
query='tl_id = 697'
list=dbfind(query)
dbext,list,'stud_acr,obstitle,obs_dec',stud_acr,obstitle,obs_dec
dbclose
print,stud_acr,obstitle,obs_dec
\end{verbatim}

You should find the following output:

\begin{verbatim}
PRY_loop_footpoints                     
Cool lines for study of loop footpoints (e.g., near sunspot); 80" r&            
Cool lines for study of loop footpoints (e.g., near sunspot); 80" r&                                
\end{verbatim}

The latter two outputs are fields intended to give information about
the observation that are written by the EIS Chief Observer at the time
the EIS plan is created.

\subsection{Extracting detailed raster properties}

Some properties of a raster are not stored in the
\verb|eis_experiment| catalog. Examples include exposure time(s) and
the wavelength information for the data windows. These can be
extracted from the raster and line list catalogs in the case of
exposure times and wavelengths, respectively.

An example for how to extract the exposure times defined for a raster
is given below. The key link to the \verb|eis_experiment| catalog is
the raster ID.

\begin{verbatim}
dbname=getenv('SSW')+'/hinode/eis/database/planning_db/technical_db/eis_raster_db
dbopen,dbname
query='id = 46'
list=dbfind(query)
dbext,list,'exposures',exposures
dbclose
print,exposures
\end{verbatim}

The raster ID here belongs to the raster of the PRY\_loop\_footpoints
study. The output \verb|exposures| is an 8 element integer array that gives
exposure times in milliseconds. The first entry is 30000 (30~s) and
the rest are zeros. This indicates that there is only one exposure for
this raster (a maximum of 8 are possible). The Solarsoft routine
\verb|eis_get_exposure_info| makes use of the raster catalog to
extract a number of pieces of information relating to exposure time,
and the reader is referred to this for more details.

For the line list catalog, each entry has an ID number that
corresponds to the line list ID number (\verb|ll_id|) in the raster
catalog. For the raster considered above, one has:

\begin{verbatim}
dbname=getenv('SSW')+'/hinode/eis/database/planning_db/technical_db/eis_raster_db
dbopen,dbname
query='id = 46'
list=dbfind(query)
dbext,list,'ll_id',ll_id
dbclose
print,ll_id
\end{verbatim}

We find the line list ID is 15, so we can now query the line list
catalog to find the number of wavelength windows for the raster:

\begin{verbatim}
dbname=getenv('SSW')+'/hinode/eis/database/planning_db/technical_db/eis_linelist_db
dbopen,dbname
query='id = 17'
list=dbfind(query)
dbext,list,'n_lines',n_lines
dbclose
print,n_lines
\end{verbatim}

The number of wavelength windows used by this raster is thus
20. Use the \verb|dbhelp| routine to find the full list of items in
the raster and line list catalogs.

\section{The EIS\_OBS\_STRUCTURE routine}\label{sect.eos}

The routine \verb|eis_obs_structure| (available in Solarsoft) serves as an example of how to
extract information from the four databases described in this
document. There are two main options for calling it. The first is a
search for all observations between two times:
%
\begin{verbatim}
IDL> str=eis_obs_structure(t0,t1)
\end{verbatim}

The second is a search for a particular raster (not study) acronym:
%
\begin{verbatim}
IDL> str=eis_obs_structure(rast_acr='cool_lines')
\end{verbatim}
%
If you know your study acronym, then you can find the corresponding
raster acronym  by running \verb|eis_xstudy| and searching for the
study. Click on the study, then the bottom of the widget will show the
raster acronym. Generally there is a one-to-one correspondence between
rasters and studies, but multiple studies can use the same raster. You
can add the additional keyword study\_acr to pick a specific study in
the latter case. You can also use T0 and T1 to restrict the search to
a particular time range.

\verb|eis_obs_structure| identifies all
rasters that were run during this time range and extracts meta-data,
which goes into the output structure 'str'. The tags of this structure
for one example are given below:
%
{\small
\begin{verbatim}
DATE_OBS   STRING    '2007-01-20T00:27:12.000'
DATE_END   STRING    '2007-01-20T00:31:40.000'
XCEN       FLOAT           175.542
YCEN       FLOAT           76.9085
FOVX       FLOAT           40.9344
FOVY       FLOAT           304.000
FILENAME   STRING    'eis_l0_20070120_002712.fits'
TL_ID      LONG               677
STUD_ACR   STRING    'PRY_fast_dens'
RAST_ACR   STRING    'Quick_AR_raster'
LINK       STRING    'http://tcrb.nrl.navy.mil/~iuu/eis/thumbnails/rasters/quick_ar_raster/eis'...
RAST_ID    INT             44
WAVELNTH   STRING    '19282 19512 20204 20383 25632'
WAVEMIN    STRING    '19246 19476 20177 20347 25587'
WAVEMAX    STRING    '19317 19547 20230 20418 25676'
OBSTITLE   STRING    'High cadence sit-and-stare in ARs; 5s exposure; Fe XIII dens; 40" c'
OBS_DEC    STRING    'High cadence sit-and-stare in ARs; 5s exposure; Fe XIII dens; 40" c'
HOP_ID     INT              0
SCI_OBJ    STRING    ''
SLIT_WIDTH INT              1
SIT_STARE  BYTE         0
STEP_SIZE  FLOAT           1.00000
SLIT_INDEX INT              0
NSTEPS     INT             40
NEXP       INT              1
EXP_TIMES  STRING    '5.0'
\end{verbatim}
}

\section{The uniqueness of TL\_ID}\label{sect.tlid}

Each \emph{study} that is placed on the timeline is assigned a unique
timeline ID, or TL\_ID. As the mission progresses, the TL\_ID numbers
increase, although not necessarily monotonically as the EIS CO does
not necessarily create the EIS plan by systematically adding studies
from the beginning of the timeline onwards.

If you know the TL\_ID, then you can retrieve information about the
rasters by doing:
%
\begin{verbatim}
IDL> str=eis_obs_structure(tl_id=tl_id)
\end{verbatim}
%
Note that there may be multiple entries as a single study can consist
of multiple rasters.

There are three known problems with searching on the timeline ID,
however, and these are discussed below.

\subsection{Rasters before 23 November 2006}

The EIS planning tool began being used from 13:14 UT on 23 November
2006. Prior to this EIS observations exist and were assigned TL\_ID
values that were generally quite high, but the planning tool does not recognise their
existence. Therefore eventually as time passed, the planning tool
reached these TL\_ID values and reused them. An example is
TL\_ID=37200. Therefore if you search for one of these TL\_ID values
then you will find two entries widely separated in time.

For the \verb|eis_obs_structure| routine, the pre-23 November studies
are automatically filtered out, but you will see them if you use the
UIT database software.

\subsection{Response studies}

Response studies are special cases that are assigned TL\_ID values of
1, 3 and 4. If you search on one of these values, then you will find
many entries widely spaced in time. Please check the website below for
more details.

\begin{verbatim}
http://solarb.mssl.ucl.ac.uk:8080/eiswiki/Wiki.jsp?page=TriggerStudies
\end{verbatim}


\subsection{TL\_ID values 16541 to 16603 (March 2010)}

There is a period running from 07:33 on 20-Mar-2010 to 02:02
27-Mar-2010 for which the TL\_ID values were duplicated. This is
because the EIS CO who began planning at 11~UT 27-Mar-2010 did not
have the most recent copy of the database, and so the planning tool
began assigning TL\_ID values that corresponded to the start of the
previous week.

The consequence is that most of the TL\_ID values between 16541 and
16603 are duplicated. Searching for one of these TL\_ID values will
thus lead to two entries, separated by about a week.

\newpage
\appendix

\section{Document modification history}

\emph{Version 2.2}: Expanded information about
\verb|eis_obs_structure| in Sect.~\ref{sect.eos}.\\
\emph{Version 2.1}: Added Sect.~\ref{sect.asplanned}.\\
\emph{Version 2}: Added Sect.~\ref{sect.tlid}.


\end{document}
