Inconsistent flux for 2019-11

[aaust]

The plot_flux_ccdb.py returns wrong values for the 2020 data if it is executed for the full run range 71350 - 73266. Only when running the script on individual runs or the 12 batches separately, one obtains the correct number of photons and the flux-normalized yields look smooth.

I assume the CCDB calib time has to be adapted while looping through the run range.

[aaust]

This is the effect on the flux-normalized yields:

The blue curve is fairly smooth, except some edges at the borders of the microscope.

[aaust]

I also noticed two related issues:

1. The flux for 2017-01 REST ver03 and ver04 comes out exactly the same, even though the calibration time stamps used for the energy table are different:

python2.7 plot_flux_ccdb.py -b 30274 -e 31057 -n 250 -m 2 -x 12 -r 3
RCDB quergy = @is_production and @status_approved
CCDB calibtime for energy to match REST ver03 = 2018-07-27-17-14-00

vs

python2.7 plot_flux_ccdb.py -b 30274 -e 31057 -n 250 -m 2 -x 12 -r4
RCDB quergy = @is_production and @status_approved
CCDB calibtime for energy to match REST ver04 = 2020-07-24-00-00-00

2. The flux for 2017-01 is consistent with analysis launch ver46, but not with analysis launch ver60.

In between, the reverse lookup was introduced for the ReactionFilter to figure out the counter number from the stored photon energy: JeffersonLab/halld_recon#536

[jrstevenjlab]

Thanks @aaust. There are few issues to address here

1. For the 2020 data there were two separate calibration time stamps used for the Spring and Summer 2020 periods and plot_flux_ccdb.py determines the CCDB timestamp from the begin_run command line option. So at the moment attempting to get the flux for the Spring and Summer 2020 periods combined will use the wrong CCDB timestamp for the Summer 2020 period. To fix this I need to have the code check if the CCDB timestamp should be updated for each new run number

2. The flux being the same between 2017 REST I believe is due to same timestamp being to retrieve the flux tables. For the REST ver03, we want to use an older timestamp from before the REST ver04 fluxes were uploaded. Since they both have the same (default) CCDB context, the only way to differentiate them is by the timestamp

3. The last issue of the mismatch with Analysis Launch ver60 is that we should be using the new calibration of the photon energy when computing the flux, but the code only uses the new calibration if the associated tables exist, with the existing REST ver03 timestamp those tables are not in CCDB. To fix this we need to have a flag for users to specify their RunPeriod and Analysis Launch version, so we can choose the correct tagger energy calibration.

These will each take some time to sort out and determine the best place to keep track of this information, either in a new DB table or hard coding the relevant transitions in this script.

[jrstevenjlab]

Thanks to @aaust for adding some additional tables to our dataVersion DB https://halldweb.jlab.org/cgi-bin/data_monitoring/monitoring/dataVersions.py, we now have calibration times available for the relevant analysis launches that use the improved tagger energy calibration.

An updated version of plot_flux_ccdb.py is on a branch
https://github.com/JeffersonLab/hd_utilities/compare/plot_flux_ver1.0

Note: this now requires the user to provide the REST and Analysis Launch version of the ROOT trees they're using for analysis since the assuming the defaults is no longer sufficient.