DRAGON Public DRAGON Private Forum Experiments Documentation Operator Information Contact Information
Using NOVA for Data Analysis

Contents:


History

NOVA is an interpretive data analysis package written by Peter Green in the 1990s in support of experiments at TRIUMF and elsewhere. It accepted a number of data formats for analysis on a number of now-defunct hardware and operating systems.

Its interpretive system allows modification or addition to calculated variables, tests and histograms "on the fly" without having to edit and recompile source code. Tables defining the variables, tests and histograms in concise form may be saved along with the histograms themselves. The penalty is slower rate of analysis than with compiled systems: NOVA is good for exploratory phases of data analysis, less good for production mode.

With the passage of time and reduction in support, NVOA now operates only on isdaq07 computer at TRIUMF for DRAGON data recorded in MIDAS ".mid" files. Dave Hutcheon, consulting as needed with Konstantin Olchanski, attempts to deal with changes in data format and updates to linux file management and diplay packages.


How to run NOVA on isdaq07

  1. Login to isdaq07 under the "dragon" account.

  2. Type:

    >cd dave/nova
    >source novanova2
    >novamap

  3. This should produce a list of running NOVA processes.   If there are no processes or none is suitable, you must start one. Type "cd src_exe" and look for subdirectories of the sort " 2013mar24"; the most recent one should match the current hardware configuration. Go to that directory and type "novastart myname" which will start and map you to a NOVA process "myname".

  4. Go to the directory in which you want to save histograms and tables. Type "nova".  If it was a new NOVA process, you will be told "All tables are empty". For some configurations there may be a suitable .nbc file in home/dragon/dave/nova/replayrunxxxx.nbc where xxxx is the run number.) Type "status". This will display status information, including run number and the most recent source of data.  Hit any key to exit this page.

  5. NOVA can accept many different formats of data.   To ensure the right use of input data, type the following:

    >close   [don't worry if told "already closed"]
    >imode dragon   [selects the input mode]

  6. To replay a .mid file, the Open statement would look something like

    >open S1318/run05050.mid, where S1318 is a link defined in the directory from which the NOVA process was launched (see 3 above). E.g. S1318-> /isdaq/data7/dragon/S1318/ (An "upgrade" of the file management system destroyed the earlier feature by which NOVADATADIR could be set to the desired path.) When a file has been replayed to end-of-file (as shown on the Status page), it is necessary to type "close" before you can open another file (or re-open the same file.)

  7. NOVA commands are keyboard-based. There is one graphics window for displaying histograms. To tell NOVA what type of display it is dealing with, type "set terminal xw" for an x-window. Do this before trying to display a spectrum using the "dsp" command.

    See also the NOVA section in the DAQ group's page on the TRIUMF website for in-depth discussion of NOVA features.


General information about NOVA

Some of the frequently-used commands are:

zero [name] [zero the value of the histogram or variable called "name"]

zall [zero all histograms and pass/fail counters]

ea ['enable acquisition' - start analysis]

da ['disable acquisition' - stop analysis]

dsp [histogram name or histo-group name] [display histogram(s)

Consult 'help dsp' for some of the useful arguments which can be used for display control, such as /max=..., /xl=..., /xh=... /xp, /yp, /log]


list [name] [give a short description of the thing called "name"]

list/full [name] [give a full description of it]

list/ref [name] [list all the references to "name" in other definitions]

list/op n1 n2 [list the Operation Sequence, lines n1 through n2]

list/if [list the pass/fail statistics for the IF tests in the Op.Seq.]

dump [filename] [saves the tables, Operation Sequence and histograms in a file called name.nbc   (for Nova Binary Compressed)]

dump/tables [name] [saves tables, Op. Seq. but not contents of histograms]

load [filename] [replaces current tables/Op Seq/histograms with those stored in file "filename"]

load/add [filename] [adds the histogram contents of "filename" to the current values of the histograms]

ha/remote [makes hardcopy of the graphics window on the printer. For an HP laserjet, precede this by "set printer hppcl"]

cur [activates "cursor mode".  Enter this separate little world by mouse-clicking when the cursor is in the graphics window; but remember, the only way to get out of this world is to type 'q' and then click back on the NOVA dialog window.

Useful commands:

l = defines the left limit of a region
r = defines the right region of a limit
s = calculates sum/centroid/FWHM of the region
e = expands between Left and Right limits
f = display the full range of the histogram
d = update the display ('draw')
q = 'quit' cursor mode]


stat [shows a Status page; hit any key to exit]

help [lists commands for which info is available]

exit [exit NOVA (surprise!)]

The following is an illustration of how the "starter" tables are set up. As the name states, the Operation Sequence defines the series of actions to be carried out when an event is presented for analysis. Type "list/op" and get the following listing of an Operation Sequence.

 
   0  ( 0) $BEG_OPSEQ:     CONTINUE 
   1  ( 0)                 IF (EPCS)THEN 
   2  ( 1)                  INCR EPICSTRIPS 
   3  ( 1)                   IF (CMD1NMR)THEN  
   4  ( 2)                     EVAL MD1CLR  
   5  ( 2)                     EVAL MD1SET  
   6  ( 2)                   ENDIF  
   7  ( 1)                   IF (CED1)THEN  
   8  ( 2)                     EVAL ED1CLR  
   9  ( 2)                     EVAL ED1SET  
  10  ( 2)                   ENDIF  
  11  ( 1)                 ENDIF  
  12  ( 0)                 IF (SCTD)THEN  
  13  ( 1)                   EVAL TRATES  
  14  ( 1)                   INCR TCHARTS  
  15  ( 1)                   EVAL ELAST_CLR  
  16  ( 1)                 ENDIF  
  17  ( 0)                 IF (SCHD)THEN  
  18  ( 1)                   EVAL HRATES  
  19  ( 1)                   INCR HCHARTS  
  20  ( 1)                 ENDIF  
  21  ( 0)                 IF (TSCH)THEN  
  22  ( 1)                   EVAL HTSC  
  23  ( 1)                   IF (HTDC0)THEN  
  24  ( 2)                     EVAL HRF1190  
  25  ( 2)                     EVAL HXO1190  
  26  ( 2)                     EVAL HV1190  
  27  ( 2)                   ENDIF  
  28  ( 1)                   IF (HADC0)THEN  
  29  ( 2)                     EVAL HV792  
  30  ( 2)                     EVAL HNTH792  
  31  ( 2)                   ENDIF  
  32  ( 1)                   IF (VTRH)THEN  
  33  ( 2)                     EVAL HVTR  
  34  ( 2)                   ENDIF  
  35  ( 1)                   IF (CHXOVER)THEN  
  36  ( 2)                     IF (HSTORE==1)THEN 
  37  ( 3)                       INCR HD_TL_CO  
  38  ( 3)                       IF (CGOLDEN)THEN  
  39  ( 4)                         OUTPUT HSINGBUF  0  
  40  ( 4)                         OUTPUT TSINGBUF  0  
  41  ( 4)                       ENDIF  
  42  ( 3)                     ENDIF  
  43  ( 2)                     IF (HSTORE==2)THEN  
  44  ( 3)                     ENDIF  
  45  ( 2)                     INCR HD_CO  
  46  ( 2)                   ENDIF  
  47  ( 1)                   INCR HD_STAMP  
  48  ( 1)                   INCR HD_SING_T 
  49  ( 1)                   INCR HD_SING_E  
  50  ( 1)                   IF (CHPULSER)THEN  
  51  ( 2)                   ENDIF  
  52  ( 1)                 ENDIF  
  53  ( 0)                 IF (TSCT)THEN  
  54  ( 1)                   EVAL TTSC  
  55  ( 1)                   IF (TLT0)THEN  
  56  ( 2)                     EVAL TV1190  
  57  ( 2)                     EVAL TRF1190  
  58  ( 2)                   ENDIF  
  59  ( 1)                   IF (TLQ0)THEN  
  60  ( 2)                     EVAL TADC0  
  61  ( 2)                   ENDIF  
  62  ( 1)                   IF (TLQ1)THEN  
  63  ( 2)                     EVAL TADC1A  
  64  ( 2)                     EVAL TADC1B  
  65  ( 2)                   ENDIF  
  66  ( 1)                   IF (VTRT)THEN  
  67  ( 2)                     EVAL TVTR  
  68  ( 2)                   ENDIF   
  69  ( 1)                   IF (CTXOVER)THEN 
  70  ( 2)                     IF (TSTORE==1)THEN  
  71  ( 3)                       EVAL TTTRIG  
  72  ( 3)                       INCR HD_TL_CO  
  73  ( 3)                       IF (CGOLDEN)THEN  
  74  ( 4)                         OUTPUT HSINGBUF  0  
  75  ( 4)                         OUTPUT TSINGBUF  0  
  76  ( 4)                       ENDIF  
  77  ( 3)                     ENDIF  
  78  ( 2)                     IF (TSTORE==2)THEN  
  79  ( 3)                     ENDIF  
  80  ( 2)                     INCR TL_CO  
  81  ( 2)                   ENDIF  
  82  ( 1)                   EVAL TTRIG_CLR  
  83  ( 1)                   EVAL TTRIG_COPY  
  84  ( 1)                   INCR TL_STAMP  
  85  ( 1)                   IF (CTPULSER)THEN  
  86  ( 2)                   ENDIF  
  87  ( 1)                   INCR TL_SING_T  
  88  ( 1)                   INCR TL_SING_E0  
  89  ( 1)                   INCR TL_SING_E1  
  90  ( 1)                   IF (CESB0)THEN  
  91  ( 2)                     EVAL ELAST_INC  
  92  ( 2)                   ENDIF  
  93  ( 1)                 ENDIF  
  94  ( 0) $END_OPSEQ:     CONTINUE  
Nova>

The quantities "EPICS", "SCTD", "TSCH", etc. are Conditions which can take on values True (represented by -1) or False (represented as a 0).  As for Fortran, C, etc., these IF statements control the flow of data analysis. The usual operation is INCR, meaning increment the named histogram (or group of histograms).   NOVA will do whatever calculations it needs to increment the named histos; unneeded calculations are not done. It is possible to force the evaluation of a variable by inserting an EVAL command in the Op Seq.

The command "list/if" gives only the IF statements with their pass/fail percentages. Note the listing of line numbers and level of nesting, also.

   1  ( 0)                 IF (EPCS)THEN
                     (   3587487         0   0.0%)
   3  ( 1)                   IF (CMD1NMR)THEN
                       (         0         0   0.0%)
   7  ( 1)                   IF (CED1)THEN
                       (         0         0   0.0%)
  12  ( 0)                 IF (SCTD)THEN
                     (   3587487      3051   0.1%)
  17  ( 0)                 IF (SCHD)THEN
                     (   3587487      3301   0.1%)
  21  ( 0)                 IF (TSCH)THEN
                     (   3587487    281947   7.9%)
  23  ( 1)                   IF (HTDC0)THEN
                       (    281947    281947 100.0%)
  28  ( 1)                   IF (HADC0)THEN
                       (    281947    281947 100.0%)
  32  ( 1)                   IF (VTRH)THEN
                       (    281947    281947 100.0%)
  35  ( 1)                   IF (CHXOVER)THEN
                       (    281947      6366   2.3%)
  36  ( 2)                     IF (HSTORE==1)THEN
                         (      6366      3475  54.6%)
  38  ( 3)                       IF (CGOLDEN)THEN
                           (      3475        17   0.5%)
  43  ( 2)                     IF (HSTORE==2)THEN
                         (      6366         0   0.0%)
  50  ( 1)                   IF (CHPULSER)THEN
                       (    281947         0   0.0%)
  53  ( 0)                 IF (TSCT)THEN
                     (   3587487   3292610  91.8%)
  55  ( 1)                   IF (TLT0)THEN
                       (   3292610   3292610 100.0%)
  59  ( 1)                   IF (TLQ0)THEN
                       (   3292610   3292610 100.0%)
  62  ( 1)                   IF (TLQ1)THEN
                       (   3292610   3292610 100.0%)
  66  ( 1)                   IF (VTRT)THEN
                       (   3292610   3292610 100.0%)
  69  ( 1)                   IF (CTXOVER)THEN
                       (   3292610     28749   0.9%)
  70  ( 2)                     IF (TSTORE==1)THEN
                         (     28749      1108   3.9%)
  73  ( 3)                       IF (CGOLDEN)THEN
                           (      1108         6   0.5%)
  78  ( 2)                     IF (TSTORE==2)THEN
                         (     28749         0   0.0%)
  85  ( 1)                   IF (CTPULSER)THEN
                       (   3292610         0   0.0%)
  90  ( 1)                   IF (CESB0)THEN
                       (   3292610    376188  11.4%)

To find out what the meaning of the Conditions is, we "list" them:

Nova>list schd
  SCHD            Condition 0              Try 3587487  Pass 3301      (  0.1 %)
    $RAW4(0)==CH_SCHD
 Test if Head scalers event
Nova>list/hex ch_schd
  CH_SCHD         I*4 Param  '44484353'X
Nova>

This tells us that Condition "SCHD" is True when the first (zero-th) element of the "raw" data of an event from the MIDAS .mid file matches the bit pattern of CH_SCHD. These events are from the scalers at the Head end of the DRAGON detector electronics (as indicated in the optional description line.)

 
Nova>li htsc
  HTSC            I*4 Var    0
    USR08(HTSC,32,$RAW4,HTSCHDPTR,HTSCHLEN,HSINGBUF,HTSCHXBEG,0,HTSCHXLEN,0)
 Put io32 data into first 'htschlen' words of hsingbuf
Nova>

This calls the user-defined subroutine USR08. Its purpose is to unpack data in the MIDAS event coming from the IO32 module and copy it into a "Head end singles" buffer HSINGBUF. The arguments include pointers and length counts. Normally, the user need not be concerned with the details of USR08 (whose source code is in the directory dave/nova/src_exe); basic informaion about the current version of a user subroutine can be obtained by asking "which":

 
Nova>which usr08
USR08 Decode Caen VME modules ... d.a.h. 4Jun2014

Digging deeper, we find test HTDC0 to see if we now are pointing to the start of data from TDC0. (The value 30 (hexadecimal) codes '0', 43 is 'C', 44 is 'D', 54 is 'T')

 
Nova>li htdc0
  HTDC0           Condition -1             Try 281947   Pass 281947    (100.0 %)
    $RAW4(HTDC0PTR)==CH_TDC0
 Test if Head TDC0 sub-event
Nova>li/hex ch_tdc0
  CH_TDC0         I*4 Param  '30434454'X

It it is TDC0 data, three 'EVAL' commands force the copy of data for RF timing, Crossover timing from the Tail electronics and other V1190 data to HSINGBUF.

After unpacking data for other Head modules, there is a test CHXOVER that checks if this is a part of a Head-Tail coincidence event, as indicated by "Crossover" TDC data. If it is part of a coincidence event, a test on HSTORE tests to see whether the corresponding data from the Tail has already been unpacked. If so, a group of Head-Tail coincidence histograms are incremented (INCR HD_TL_CO) and then groups of singles Head histograms (HD_STAMP, HD_SING_T, HD_SING_E)

(Since 2013 the DRAGON data acquisition system has operated as independently running Head and Tail electronics, with synchronized Time Stamps as the way to match up the two components of a gamma-recoil coincidence event. Other versions of novamain.exe are needed to analyze MIDAS data from before 2013; look in the src_exe directory for files such as "novamain_feb1105.exe").

To see exactly what is being incremented, say by INCR HD_SING_E at line 49 in the Op Seq, type "list/mem hd_sing_e":

 
  HD_SING_E       Spectrum Group with 6 members
  HG0_NG0_SING    2D Spec   (200,30)       Try 281947   Pass 281947    (100.0 %)
  HE0_G_SING      1D Spec    (500)         Try 281947   Pass 281947    (100.0 %)
  HZG0_SING       1D Spec    (40)          Try 281947   Pass 75010     ( 26.6 %)
  HE2_NG2_SING    2D Spec   (100,30)       Try 281947   Pass 281947    (100.0 %)
  HE0E1_SING      2D Spec   (50,50)        Try 281947   Pass 281947    (100.0 %)
  HHPEDHUNT       1D Spec    (200)         Try 281947   Pass 281947    (100.0 %)

And with

Nova>li hg0_ng0_sing
  HG0_NG0_SING    2D Spec   (200,30)       Try 281947   Pass 281947    (100.0 %)
    E0_G (1000.0000,8000.0000)                NG0 (0.0000,30.0000)
     (0 Conditions)

we learn that the histogram with the name HG0_NG0_SING (12 characters are the maximum that will be recognized by NOVA) is a 2-dimensional, 200 bins by 30 bins, with 'x' variable E0_G displayed for values between 1000 and 8000 versus 'y' variable NG0 from 0 to 30. No further conditions are tested before incrementing it.

Almost down to bedrock! E0_G and NG0 are defined by

 Nova>li ng0 e0_g pe0_g
  NG0             I*4 Var    7
    HSINGBUF(HADC0XBEG)
  E0_G            Real Var   1568.2200
    GGAIN(NG0)*(HSINGBUF(PE0_G)-GPED(NG0))
  PE0_G           I*4 Var    12
    HADC0XBEG+HADC0XCMP
 Pointer to most-energetic  BGO energy

NG0 is the number of the BGO gamma detector that registered the largest gamma-ray energy (E0_G) and PE0_G points to its location within the Head singles buffer. GGAIN and GPED are arrays of constants for the gains and pedetals of the 30 BGO detectors

I have tried to pick names which suggest the meaning of the histogram, condition, parameter or variable: most Condition names start with C, most histogram names with H.  However, this is purely optional -- you can make names as obscure as you wish!  Their true meaning can be tracked down by "list"ing.  Conversely, you can work 'up the food chain' by use of "list/ref".

Making changes


  1. To an existing Parameter

    A Parameter is a quantity which NOVA knows is constant from one event to the next (so it needn't be re-evaluated every event).   For example, HNS_PER_CHAN might mean the number of nanoseconds per channel for the heavy ion Tail TDC; if the hardware was changed so that the new value was 0.2 ns/channel, this Parameter could be updated simply by typing "hns_per_chan=0.2".

  2. To an existing Variable or Condition

    As for a Parameter, you can change the definition simply by typing in the new expression for it.  If you want to make only a small change in a long definition you can EDit it: "ed chnhits"

  3. To the Operation Sequence

    Type "ed/op" and whatever editor was specified by "setenv EDITOR" will be active.   When you exit, do not change the default name of the Op Seq file.   NOVA will take care of the indentation for any new "IF...THEN" nesting; if you give just the name of a histogram, NOVA will add the "INCR" for you.

  4. Creating a new variable

    Declare the variable type, e.g. "i*2 bgo" says that "bgo" is to be a 2-byte integer, "real h_energy" defines a floating-point variable. (If you omit the declaration, NOVA will assume the Fortran convention of Integer or Real according to the first letter of the name.) Once declared, the variable is given its definition, e.g. "bgo = $raw(21)" would equate "bgo" to element 21 in the vector of raw data.   [Unfortunately, DRAGON data come in variable-length events, so this would not be a useful definition, except for Scaler events.] Variables can refer to other defined variables.  One tricky, but useful, definition refers to Conditions: because a Condition takes on the value -1 if True, 0 if False, "esum = -ce1*e1 - ce2*e2" will take on the value e1 if condition ce1 is True, ce2 is False; if both ce1 and ce2 are False, esum=0 while if both are true esum=e1+e2.

  5. Creating a new Condition

    Declare the Condition (e.g. Condition spike), then define it: e.g. "spike = INSIDE(5,bgo,7)).   (A 'feature' of the INSIDE test is that if "bgo" had been a Real quantity, the lower and upper limits would have to have included a decimal point.)

  6. Creating/modifying a 1-d histogram

    "def1d/xs=200/xl=2.5/xh=4.5/xd=e_recoil hhe" would create a histogram named "hhe" having 200 bins for the variable "e_recoil", between lower limit 2.5 and upper limit 4.5.  (If neither /xl nor /xh had been supplied, they would have been defaulted to /xl=0 and /xh=200 (the number of bins).) This defines the histogram, but it will not be incremented until either "INCR HHE" appears in the Op Seq or HHE is added to a Group that is incremented. The lower or upper limits or the variable of an existing histogram may be varied by, for example, "def1d/xl=7.2/xh=8.8 hhe" to change the lower and upper limits of histogram HHE.  Note that the size (number of bins) of a histogram may not be changed in this way; you must "Delete" the histogram and make a new def1d with the desired argument for /xs.

  7. Creating/modifying a 2-d histogram

    This is just as for the 1-d histograms, except that additional arguments /ys, /yl, /yh and /yd are called for.

  8. Tips

    • "list/und" will tell you about any undefined quantities.   This usually happens when you mis-spell something.
    • "list/orph" will identify anything which has been defined but which is never used.
    • You can open a shell from NOVA by typing "dcl".   Close the shell and return to NOVA by "exit"

Software Scalers


The variable TRATES is a vector and also invokes the user subroutine USR00, that extracts Head scaler rate information from the MIDAS data stream and also updates TTOTS, that integrates the TRATES data to record Total counts (since it was last zeroed).  

EPICS data
EPICSTRIPS is a group of histograms of the 'chart' type, designed to look like an analog strip chart.

  Nova>li/mem epicstrips
  EPICSTRIPS      Spectrum Group with 8 members
  HPCELL          1D Chart   (2000)        Try 0        Pass 0         (  0.0 %)
  HFC1            1D Chart   (2000)        Try 0        Pass 0         (  0.0 %)
  HTCELL          1D Chart   (2000)        Try 0        Pass 0         (  0.0 %)
  HMD1NMR         1D Chart   (200)         Try 0        Pass 0         (  0.0 %)
Nova>li htcell tcell /hex ep3 ctcell
  HTCELL          1D Chart   (2000)        Try 0        Pass 0         (  0.0 %)
    TCELL (0.0000,2000.0000)
     (CTCELL)
 Gas target cell temperature
  TCELL           Real Var   1.9948
    USR10(TCELL,EP3)
 Temperature in target gas cell
  EP3             I*4 Var    'B09792F0'X
    $RAW4(3)
  CTCELL          Condition '0'X           Try 0        Pass 0         (  0.0 %)
    $RAW(5)==16448
Nova>which usr10
USR10 I*4 to Real conv. ... d.a.h. 25 Apr 2013
 

Note: each element of $RAW4 contains 4 bytes of 'raw' data, while $RAW has the same data in 2-byte lements. I.e. $RAW4(3) data are also in $RAW(6)/$RAW(7). CTCELL picls out the EPICS channel assignment for Cell Temperature. Unfortunately, there was a reassignment of EPICS channel numbes that this NOVA file didn't know about. Consult with the DRAGON DAQ expert to figure out current EPICS channels.

Lines 3-6 of the Operation Sequence are a way to make the MD1NMR variable (which is read out periodically via EPICS) available for calculations involving Tail events: Tail events have no EPICS data and so a variable MD1NMR would be zero. To work around this ussue, a user subroutine was created to transfer the value of the Variable to a (fixed) Parameter by EVAL command.

 Nova>li md1clr md1set
  MD1CLR          I*4 Var    0
    USR04(MD1CLR,MD1PAR,0)
  MD1SET          I*4 Var    0
    USR04(MD1SET,MD1PAR,MD1NMR)
Nova>which usr04
USR04 Incr/decr/clear an I*4    d.a.h. 12Jun01

MD1PAR makes available in analysis of Tail (or Head) events the value of MD1NMR as of the most recent EPICS readout.


Dave Hutcheon
25 January 2002
13 March 2019 update