PRO xrt_tim2xobt, $ time_in, $ archive_prefix=archive_prefix, $ xob_start =xob_start, $ xob_stop =xob_stop, $ xob_string =xob_string, $ start_file =start_file, $ stop_file =stop_file, $ timeline_dir =timeline_dir, $ qerror =qerror, $ qstop =qstop ;================================================================== ;+ ; ; PROJECT: ; Solar-B / Hinode / XRT ; ; NAME: ; XRT_TIME2XOBT ; ; CATEGORY: ; ; XRT Data Verification ; ; PURPOSE: ; ; Given a time, identify the XOB description, and start and ; stop times. Determines this information from COT's dr_XRT*.sim ; files, with all attendant caveats. ; ; CALLING SEQUENCE: ; ; XRT_TIM2XOBT, TIME_IN [,ARCHIVE_PREFIX=ARCHIVE_PREFIX] $ ; [,XOB_START=XOB_START] [,XOB_STOP=XOB_STOP] $ ; [,XOB_STRING=XOB_STRING] [,START_FILE=START_FILE] $ ; [,/VERBOSE] [,/QSTOP] ; ; INPUTS: ; ; TIME_IN - A timestamp that pins the point on which to identify ; the encompassing XOB. Date and time may be ; in any format that is recognized by ANYTIM.PRO . ; ; ARCHIVE_PREFIX - [OPTIONAL] A string that specifies the directory ; location of the local XRT archive. E.g., at SAO, ; archive_prefix='/archive/hinode/xrt/'. ; (Default = '/archive/hinode/xrt/'.) ; TIMELINE_DIR - [OPTIONAL] Set location of timeline directory ; ; /QSTOP - [OPTIONAL] Tell program to stop before exiting. ; Just in here for debugging. ; ; ; OUTPUTS: ; ; XOB_START - Start time for the XOB that spans TIME_IN. ; ; XOB_STOP - Start time for the XOB that starts AFTER ; TIME_IN. This is used as a proxy for the stop ; time corresponding to XOB_START. It is entirely ; possible that the XOB actually stops well ; before this time. Also note that XOB_STOP ; exactly includes the first image from the next ; XOB (according to the dr_XRT*.sim file), so ; it should be treated as an excluded boundary. ; ; XOB_STRING - A long string that describes the XOB, including ; its title. This is taken verbatim from the ; dr_XRT*.sim file. ; ; START_FILE - Full pathname for the dr_XRT*.sim file that ; contains XOB_START. ; ; QERROR - return an error if program does not execute ; properly ; 1 - timeline directory not found ; 2 - empty timeline sim file ; ; EXAMPLES: ; ; Here is a straightforward usage: ; ; IDL> time_in = '2010-sep-6 15:00' ; IDL> xrt_tim2xobt, time_in, xob_start=xob_start, $ ; IDL> xob_stop=xob_stop, xob_string=xob_string ; IDL> help, xob_start, xob_stop, xob_string ; XOB_START STRING = '06-Sep-2010 14:18:32' ; XOB_STOP STRING = '06-Sep-2010 18:08:02' ; XOB_STRING STRING = 'program AR Standard-A(Filter-Ratio)'... ; ; ; METHOD: ; ; This program looks for the XOB start times that fall before and ; after TIME_IN. These bounding times are called T_BOUND_A and ; T_BOUND_B. Each of these might be defined and/or overwritten ; by several dr_XRT*.sim files. FILEN refers to the dr_XRT*.sim ; file with the same datestamp as TIME_IN. Adjacent dates are ; referred to by N-1, N+1, ... etc. ; ; Starting with date N (which is the date of TIME_IN), look for ; XOB start times immediately bounding TIME_IN. *IF* lower bound ; cannot be found in file, or if the N day file does not exist, start ; looking backwards for earlier files. Keep going backwards until a ; lower bound is found. This approach ensures that the most recent ; file (with an XOB that spans TIME_IN and immediately before it) ; is used. ; ; The preceeding step establishes the lower bound XOB_START. ; XOB_START (ie, the start time of the XOB following TIME_IN) *may* ; have been established in that step. *OR* it may still be unknown. ; *OR* it may be superceded in a timeline file after. So the next ; step is to go forward to dr_XRT<>.sim, and see if there ; is an XOB_STOP closer to TIME_IN. Then check dates N+2, N+3, ... ; etc. Cease checking when (a) a file is found; and (b) the first ; XOB start time in the file is later than a previously established ; XOB_STOP. (It is illustrative to consider that a timeline can ; theoretically span many days into the future, and XOB_STOP may be ; several days after TIME_IN, such that subsequent timelines could ; overwrite XOB_STOP even after TIME_IN has passed.) ; ; It is possible under this logic that XOB_START and XOB_STOP ; might be determined from different dr_XRT*.sim files. However, this ; program will use the best information from all of the dr_XRT*.sim ; files to set these values in the most accurate way. ; ; ; NOTES: ; ; 1) The data represent the outcome of a complicated operating ; environment, including OP/OG commands through COT, other OP/OG ; commands, real-time commands, safehold responses, and other ; complications. This program only refers to the dr_XRT*.sim file ; created by the COT in order to understand the start and stop times ; of XOBs, and in that sense is an imperfect reflection of the expected ; data flow. The human operator MUST use their own judgement and ; analysis to interpret the outputs of this program. ; ; ; CONTACTS & BUG REPORTS: ; ; Comments, feedback, and bug reports regarding this routine ; should be directed to this email address: ; xrt_manager ~at~ head.cfa.harvard.edu ; ; When sending a Bug Report, please be sure to include the ; following items: ; ; - Which program version used. (This is in the PROGVER variable.) ; ; - Value for TIME_IN, as given to the program. ; ; - The formulation of the call to XRT_TIM2XOBT you used. ; (I.e., show exactly how you called the program.) ; ; - A copy of the entire error message given by IDL when the ; program crashed (if it crashed). ; ; - A clear example of erroneous outputs, if that is the problem. ; ; ; MODIFICATION HISTORY: ; progver = 'v2010-Oct-01' ;--- (M.Weber, SAO) Written. progver = 'v2011-May-09' ;--- (K. Reeves, SAO) Added a check for T_BOUND_Z before ; searching forward in time for it. progver = 'v2011-May-11' ;--- (K. Reeves, SAO) Added logic that uses the last line ; of the most recent sim file for ; xob_stop if there is no subsequent file progver = 'v2011-May-18' ;--- (K. Reeves, SAO) Added logic skips a file if it ; exists but contains no times progver = 'v2011-Jul-22' ;--- (K. Reeves, SAO) Added timeline_dir and qerror ; keywords, added check that ; timeline_dir exists. progver = 'v2011-Dec-03' ;--- (K. Reeves, SAO) Added keyword to return timeline file for ; xob_stop progver = 'v2013-Sept-04' ;--- (K. Reeves, SAO) Added error code for empty dr*sim file, which ; sometimes happens during long bakeouts. ;- ;================================================================== qstop = keyword_set(qstop) default, archive_prefix, '/archive/hinode/xrt/' time_in_ints = anytim(time_in, /ints) qdone = 0b q_Afound = 0b q_Zfound = 0b examine_date = time_in lastline_sec = 0b repeat begin if qstop then stop ;; Convert the EXAMINE_DATE to a pathname for a "dr_XRT*.sim" ;; file. Go find the file. ex_datecode = date_code(examine_date) ;; set timeline_dir to SAO if keyword not set for another location if not keyword_set(timeline_dir) then timeline_dir = '/archive/hinode/xrt/timelines/' ;; check that timeline directory exists if not dir_exist(timeline_dir) then begin print, 'Timeline directory '+ timeline_dir +' not found.' print, 'Exiting...' qerror=1 return endif ex_file_pattern = timeline_dir $ + strmid(ex_datecode,0,4) + '/' $ + strmid(ex_datecode,4,2) + '/' $ + strmid(ex_datecode,6,2) + '/' $ + ex_datecode + '_exported/dr_XRT_' $ + ex_datecode + '*.sim' ex_file = find_file(ex_file_pattern,count=cnt_exfile) ;; If the file exists for EXAMINE_DATE, then look at the ;; XOB start times inside. if (cnt_exfile ge 1) then begin ;; Read file contents into a string array. text = rd_text(ex_file[0]) ;; To identify XOB start times, we look for (a) lines ;; that start with "Simulating..."; and (b) that there ;; are simulated images starting two lines later with date ;; codes. There are cases where an XOB is defined right after ;; another, and the first one won't actually have images in ;; the file. N_text = n_elements(text) twochar = strmid(text,0,2) ;; check if the sim file is corrupted or contains no times if n_elements(text) lt 3 then begin print, '' print, 'XRT_TIM2XOBT: Empty sim file detected: ' print, ex_file qerror = 2 return endif else begin twochar_base = twochar[0:N_text-3] twochar_plus2 = (shift(twochar,-2))[0:N_text-3] ss1 = where( (twochar_base eq 'Si') and $ (twochar_plus2 eq '20'), cnt1) ;; record last time in file, in case there's no following file lastline = text[N_text-1] lastline_time_here = strmid(lastline, 0,15) lastline_t_str = strmid(lastline_time_here,0,4) + '-' + $ strmid(lastline_time_here,4,2) + '-' + $ strmid(lastline_time_here,6,5) + ':' + $ strmid(lastline_time_here,11,2) + ':' + $ strmid(lastline_time_here,13,2) lastline_sec_here = anytim(lastline_t_str) ;; keep greatest lastline time if lastline_sec_here gt lastline_sec then $ lastline_time = lastline_time_here lastline_sec = lastline_sec_here ;; If there are valid XOB starting times in the file, ;; then proceed to see if any of them can be T_BOUND_A ;; or T_BOUND_Z. if (cnt1 ge 1) then begin ;; Extract the time codes as strings. ex_times = strmid(text[ss1+2], 0, 15) ;; Convert times to an array of seconds, referenced to the ;; TIME_IN we are looking for. ex_t_str = '_' + strmid(ex_times,0,8) + $ '_' + strmid(ex_times,9,6) ex_t_ints = anytim2ints(fid2time(ex_t_str,'_')) ex_t_secarr = int2secarr(ex_t_ints, time_in_ints) ;; T_BOUND_A is defined as the latest start time ;; *before* TIME_IN, so it will be negative in the ;; seconds array. ss_A1 = where(ex_t_secarr le 0, cnt_A1) if (cnt_A1 ge 1) then begin ss_A2 = where(ex_t_secarr eq max(ex_t_secarr[ss_A1]), cnt_A2) t_bound_A = ex_times[ss_A1[ss_A2[0]]] xob_start = (fid2time(ex_t_str,'_'))[ss_A1[ss_A2[0]]] xob_string = (strmid(text[ss1[ss_A2[0]]], 11, 110))[0] start_file = ex_file[0] q_Afound = 1b endif ;; T_BOUND_Z is defined as the earliest start time ;; *after* TIME_IN, so it will be positive in the ;; seconds array. ;; It is possible that we are still looking backwards for ;; T_BOUND_A even after T_BOUND_Z has been found. ;; If we keep going back looking for T_BOUND_A, then we *only* ;; update T_BOUND_Z if we find a time value *before* the ;; start time of files that have already been checked. ;; (If they were *after* the start time of a file in the future, ;; that means that the future file overwrote the plan for that ;; time period.) ss_Z1 = where(ex_t_secarr gt 0, cnt_Z1) if (cnt_Z1 ge 1) then begin ss_Z2 = where(ex_t_secarr[ss_Z1] eq min(ex_t_secarr[ss_Z1]), cnt_Z2) if (q_Zfound eq 1b) then begin if (ex_t_secarr[ss_Z1[ss_Z2[0]]] lt last_starttime) then begin t_bound_Z = ex_times[ss_Z1[ss_Z2[0]]] xob_stop = (fid2time(ex_t_str,'_'))[ss_Z1[ss_Z2[0]]] stop_file = ex_file[0] ex_t_secarr_Z = ex_t_secarr[ss_Z1[ss_Z2[0]]] last_starttime = ex_t_secarr[0] endif endif else begin t_bound_Z = ex_times[ss_Z1[ss_Z2[0]]] xob_stop = (fid2time(ex_t_str,'_'))[ss_Z1[ss_Z2[0]]] stop_file = ex_file[0] ex_t_secarr_Z = ex_t_secarr[ss_Z1[ss_Z2[0]]] last_starttime = ex_t_secarr[0] q_Zfound = 1b endelse endif ;; Stop looking backwards for T_BOUND_A and T_BOUND_Z if ;; and only if T_BOUND_A is found. if (q_Afound eq 1b) then qdone = 1b endif ;; (valid XOB start times in file) endelse endif ;; (file exists for EXAMINE_DATE) if (qdone eq 0b) then begin examine_date = xrt_dateadd(examine_date, -1L) endif endrep until qdone examine_date = xrt_dateadd(time_in, 1L) if n_elements(t_bound_z) eq 0 then qdone = 0b while (qdone eq 0b) do begin ;; only look for files that are earlier than the current date. ;; if none are found, use last line from latest file as ;; T_BOUND_Z now = anytim(systim()) if now lt anytim(examine_date) then begin t_bound_z = lastline_time t_bound_z_file = '_' + strmid(t_bound_z,0,8) + '_' + strmid(t_bound_z,9,6) xob_stop = fid2time(t_bound_z_file, '_') stop_file = ex_file[0] qdone = 1b endif else begin ;; Convert the EXAMINE_DATE to a pathname for a "dr_XRT*.sim" ;; file. Go find the file. ex_datecode = date_code(examine_date) ex_file_pattern = archive_prefix + '/timelines/' $ + strmid(ex_datecode,0,4) + '/' $ + strmid(ex_datecode,4,2) + '/' $ + strmid(ex_datecode,6,2) + '/' $ + ex_datecode + '_exported/dr_XRT_' $ + ex_datecode + '*.sim' ex_file = find_file(ex_file_pattern,count=cnt_exfile) if qstop then stop ;; If the file exists for EXAMINE_DATE, then look at the ;; XOB start times inside. if (cnt_exfile ge 1) then begin ;; Read file contents into a string array. text = rd_text(ex_file[0]) ;; To identify XOB start times, we look for (a) lines ;; that start with "Simulating..."; and (b) that there ;; are simulated images starting two lines later with date ;; codes. There are cases where an XOB is defined right after ;; another, and the first one won't actually have images in ;; the file. N_text = n_elements(text) twochar = strmid(text,0,2) if n_elements(text) lt 3 then begin print, '' print, 'XRT_TIM2XOBT: Empty sim file detected: ' print, ex_file qerror = 2 return endif else begin twochar_base = twochar[0:N_text-3] twochar_plus2 = (shift(twochar,-2))[0:N_text-3] ss1 = where( (twochar_base eq 'Si') and $ (twochar_plus2 eq '20'), cnt1) ;; If there are valid XOB starting times in the file, ;; then proceed to see if any of them can be T_BOUND_Z. ;; We assume that T_BOUND_A *cannot* occur in a file ;; starting for any day later than TIME_IN. if (cnt1 ge 1) then begin ;; Extract the time codes as strings. ex_times = strmid(text[ss1+2], 0, 15) ;; Convert times to an array of seconds, referenced to the ;; TIME_IN we are looking for. ex_t_str = '_' + strmid(ex_times,0,8) + $ '_' + strmid(ex_times,9,6) ex_t_ints = anytim2ints(fid2time(ex_t_str,'_')) ex_t_secarr = int2secarr(ex_t_ints, time_in_ints) if qstop then stop ;; T_BOUND_Z is defined as the earliest start time ;; *after* TIME_IN, so it will be positive in the ;; seconds array. ;; We need to keep looking forward at files until they ;; start later than our latest T_BOUND_Z. ;; It should be safe to assume that *all* the times ;; in this part of the search are greater than TIME_IN. ss_Z1 = where(ex_t_secarr gt 0, cnt_Z1) if (cnt_Z1 ge 1) then begin ss_Z2 = where(ex_t_secarr[ss_Z1] eq min(ex_t_secarr[ss_Z1]), cnt_Z2) if (q_Zfound eq 1b) then begin if (ex_t_secarr_Z lt ex_t_secarr[ss_Z1[ss_Z2[0]]]) then begin qdone = 1b endif else begin t_bound_Z = ex_times[ss_Z1[ss_Z2[0]]] xob_stop = (fid2time(ex_t_str,'_'))[ss_Z1[ss_Z2[0]]] stop_file = ex_file[0] ex_t_secarr_Z = ex_t_secarr[ss_Z1[ss_Z2[0]]] endelse endif else begin t_bound_Z = ex_times[ss_Z1[ss_Z2[0]]] xob_stop = (fid2time(ex_t_str,'_'))[ss_Z1[ss_Z2[0]]] stop_file = ex_file[0] ex_t_secarr_Z = ex_t_secarr[ss_Z1[ss_Z2[0]]] q_Zfound = 1b endelse endif endif endelse endif endelse if (qdone eq 0b) then begin examine_date = xrt_dateadd(examine_date, 1L) endif endwhile if qstop then stop return END