Change default calibration scaling factor tp account for different integration windows and the effect of sampling inhomogeneities

[FrancaCassol]

• Modify the global calibration scaling factor including:
  1. The factor due to the different integration window in cosmics (8 ns) and laser events (12 ns). We use the values estimated on data by Yukiho (see slide 12) : [HG,LG]=[1.088,1.004]
  2. The average estimated effect on the gain due to the DRS4 time sampling inhomogeneities and laser variation: [HG,LG]=[1.05,1.10]

• Update the excess noise factor to the most recent estimated value (Fˆ2=1.222)

It will remain a small underestimation of the light with respect to MC due to the 12 ns window, taken here as reference

[codecov]

Codecov Report

Merging #516 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #516   +/-   ##
=======================================
  Coverage   41.48%   41.48%           
=======================================
  Files          77       77           
  Lines        6501     6501           
=======================================
  Hits         2697     2697           
  Misses       3804     3804           

Impacted Files	Coverage Δ
lstchain/calib/camera/calibration_calculator.py	27.02% <ø> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e0f6a76...6e20293. Read the comment docs.

[moralejo]

• Modify the global calibration scaling factor including:
  1. The factor due to the different integration window in cosmics (8 ns) and laser events (12 ns). We use the values estimated on data by Yukiho (see slide 12) : [HG,LG]=[1.088,1.004]

@FrancaCassol @rlopezcoto
Sorry for late feedback on this: I am not sure about the factors you took from the above reference. They seem to be = (fraction of calib pulse integrated in 12 samples) / (fraction of cosmic pulse integrated in 8 samples)

I think we rather want
(fraction of cosmic pulse integrated in 12 samples) / (fraction of cosmic pulse integrated in 8 samples)

We want to integrate 8 samples in shower images to let in less noise, and then we just want to convert it (in average) to what we would get by integrating (those same pulses) in 12 samples. So I think those factors are a bit underestimated.

[FrancaCassol]

Hi @moralejo,

yes, I agree, for that in my comment I added the sentence "It will remain a small underestimation of the light with respect to MC due to the 12 ns window, taken here as reference".
We can indeed correct also for this factor, for Yukiho it should be rather fast to estimate on cosmic the missing extraction with a window of 12 ns (if I did not miss it, it is not in the slides, I can ask him).

[moralejo]

Hi @moralejo,

yes, I agree, for that in my comment I added the sentence "It will remain a small underestimation of the light with respect to MC due to the 12 ns window, taken here as reference".
We can indeed correct also for this factor, for Yukiho it should be rather fast to estimate on cosmic the missing extraction with a window of 12 ns (if I did not miss it, it is not in the slides, I can ask him).

Hi @FrancaCassol, I did not refer to the remaining pulse beyond 12 samples, but to the fact that the ratios we are using (of 12 vs. 8) are not obtained for the same pulse shapes (one is calib, the other cosmic)

[FrancaCassol]

@moralejo, I think this was the idea, they better correspond to what we have in the data ...

[FrancaCassol]

@moralejo, actually thinking it again, we even do not have the 12 ns problem: the coefficient were calculated as the ratio: extract_FF(12 ns)/extract_cosmic(8 ns)

The idea behind is :

• the gain is estimated on FF(12 ns) signals: gain= Q_FF(12 ns)/n_pe --> we correct for the 12 ns by dividing for the extract_FF(12 ns) --> gain_corrected = gain/extract_FF(12 ns)

• the cosmic charge is estimated in 8 ns: we correct it by Q_corrected = Q/extract_cosmic(8 ns)

So the overall correction factor for the coefficients dc_to_pe is :
dc_to_pe_new = dc_to_pe * extract_FF(12 ns)/extract_cosmic(8 ns)

which is what proposed...

[yukihok]

Dear @FrancaCassol and @moralejo,

Thank you very much for this important implementation.

My understanding is consistent with Franca's comment. The correction factor extract_FF(12 ns)/extract_cosmic(8 ns) would be reasonable with the current status.

Another point: The effect of DRS4 sampling inhomogeneity correction on conversion factors is ~5% for both HG and LG. So I think the correction factors for this effect would be [HG,LG]=[1.05,1.05] (not [1.05,1.10]).

[rlopezcoto]

Hi @FrancaCassol the point is that extract_FF(12 ns) is different from extract_cosmic(12 ns), which will make a slight difference in the extracted charge (coming from cosmics).
Suggested by @moralejo, I plotted the ratio of the signal in the brightest pixel for an integration of 8 ns using the default values (current standard in lstchain) and 12 ns using only the sampling correction [1.05,1.10]:

For HG LG, the mean is 0.998, so we are fine, but for LG HG, it is 0.947

[FrancaCassol]

Hi @rlopezcoto , indeed for LG I had more a value of 1.14 (instead of 1.10), but we decided to be more optimistic ;-) . Could you please try again with 1.14?

I am in any case recalculating everything with higher statistics. Those values were not meant to be totally precise, the idea was to become closer to the reality for the just coming release

[FrancaCassol]

Dear @yukihok,

Another point: The effect of DRS4 sampling inhomogeneity correction on conversion factors is ~5% for both HG and LG. So I think the correction factors for this effect would be [HG,LG]=[1.05,1.05] (not [1.05,1.10]).

These values are estimated taking into account all the systematics (proportional to the charge) and LG is globally more affected. I am going to explain the method next Monday at the LST analysis meeting. As said above, these values must be considered as provisory

[yukihok]

Dear @FrancaCassol,

I see, thank you for the explanation. Then I will wait for the next call :)

[moralejo]

@moralejo, actually thinking it again, we even do not have the 12 ns problem: the coefficient were calculated as the ratio: extract_FF(12 ns)/extract_cosmic(8 ns)

The idea behind is :

• the gain is estimated on FF(12 ns) signals: gain= Q_FF(12 ns)/n_pe --> we correct for the 12 ns by dividing for the extract_FF(12 ns) --> gain_corrected = gain/extract_FF(12 ns)
• the cosmic charge is estimated in 8 ns: we correct it by Q_corrected = Q/extract_cosmic(8 ns)

So the overall correction factor for the coefficients dc_to_pe is :
dc_to_pe_new = dc_to_pe * extract_FF(12 ns)/extract_cosmic(8 ns)

which is what proposed...

Not clear to me why you would convert the 8-sample integrated charge in cosmics pulses to 12-sample in FF pulses. I think the idea of the integration correction is to obtain the # of adc counts we would get if we integrated the cosmics pulses in 12 samples instead if 8.

[rlopezcoto]

For the record, when applying the new factor calculated in #530 , for HG and LG, the difference between

• integrating 12 ns + sampling correction and
• integrating 8 ns + full correction with the missing factor added in this PR
  is the same:
  

[moralejo]

So the overall correction factor for the coefficients dc_to_pe is :
dc_to_pe_new = dc_to_pe * extract_FF(12 ns)/extract_cosmic(8 ns)
which is what proposed...

Ok, I now think you are right @FrancaCassol, below I just try to put my thoughts together:

Let's assume for a moment the NSB noise is very low, and we can integrate as many as 20 samples.

NOTE!: Uppercase Q: cosmic charges. Lowercase q: FF charges.

We do the FF method on flatfield events, and (assuming the parameters of the method reflect reality) obtain npe as number of photoelectrons

Then we calculate gain = npe/q_20 [units: pe/adc_count] where q_20 is the average 20-sample integrated charge for FF events. This conversion factor will be ok for "whatever" t-distribution of the incoming photons, as long as all of them (and their tails) are within the 20-sample integration window.

Now we want to integrate cosmics in a smaller window, say 8 samples. But, in order to be able to use the calculated "gain", we want to extrapolate the result to the total number of adc counts we would obtain (in average, in cosmics) if we had integrated 20. So we take the average ratio <Q_20/Q_8> for cosmics pulses, and hence the new gain is corrected_gain = gain*<Q_20/Q_8> and we obtain the number of p.e. in cosmic pulses as corrected_gain*Q_8

Now, in reality, we are not calculating the FF with 20 samples, but according to Yukiho's estimates this just underestimates the number of p.e by 1.0-1.25% This slightly underestimated number is, again, npe. I will ignore the small underestimation.

Then, the gain, with the same definition as above (extrapolating to 20 samples), would be gain = npe / [q_12*<q_20/q_12>]

Now we want to use 8 samples for cosmics:

corrected_gain = gain*<Q_20/Q_8>, same as before, i.e. npe / [q_12*<q_20/q_12>)] * <Q_20/Q_8> =

npe/q_12 * <q_12/q_20> * <Q_20/Q_8> = npe/q_12 * <q_12/q_20> / <Q_8/Q_20>

I think is what you Franca and Yukiho proposed, and seems correct to me.
Then the number of p.e. in cosmics is corrected_gain*Q_8

Then Rubén checked in the data empirically the difference between integrating 8 samples (and using the above corrected gain factor) and integrating 12 samples (with no pulse integration correction). So what he did to obtain the number of pe in cosmics is
npe/q_12 * Q_12

while we should do

npe/q_12 * <q_12/q_20> / <Q_12/Q_20> * Q_12

NOTE: original entry was edited from here, following @yukihok 's correction!

For the HG he found an average number of p.e. which is 0.947 * the value with 8 samples (and the corrected gain). See @yukihok reply below. This was indeed the other way round, i.e. @rlopezcoto found the value with 12 samples (and no correction) to be a factor 1.056 higher than corrected_gain*Q_8 This would imply:

<q_12/q_20> / <Q_12/Q_20> = 1./1.056 =0.947 for the high gain

From @yukihok study, <q_12 / q_20> = 0.911
(slide 12 of https://indico.cta-observatory.org/event/2853/contributions/24597/attachments/17750/23853/ChargeRecoYukiho20200629.pdf)

which would mean: <Q_12 / Q_20> = 0.962 for the HG.
According to @yukihok 's study (see below) <Q_12/Q_20> for HG is 0.95 (note that in his notation the Q_xx values are already normalized to Q_20)

So this seems pretty consistent, and the ~1% differences might be just due to the different calculations: Ruben's is obtained as an average of the ratio for many events (brightest pixel) while Yukiho's is obtained first averaging the pulse shapes, then integrating. Many issues may be at play here: how the pulses are superimposed, how they are selected (they are bright pulses), whether the chosen pixel(s) are representative... It is not obvious to what precision the two values should agree. I tend to think that for the purpose of finding the best conversion factor the "event-wise-ratio" method id more straightforward. It tests the effect for the actual integration process we use, not for an average pulse.

Anyway, the difference is pretty small so we can ignore it for the time being - at least until we calibrate the MC regularly in exactly the same way as the real data (with simulated FF runs)

[yukihok]

Thank you very much @moralejo for getting things straight.
My proposal is exactly as you explain.

Concerning the comparison between Ruben’s analysis and mine, please let me try to clarify below and please correct me if I’m wrong. (Here, I will neglect correction factor for DRS4 correction just for simplicity.)

As far as I can see, what @rlopezcoto calculated is ratio between the following two npe values estimated in different ways.

npe_num = gain * Q_8 * correction (Here, gain = npe/q_12 and correction = q_12/Q_8 as proposed initially)
npe_den = gain * Q_12

So the calculated ratio should correspond to R = npe_num/npe_den = q_12/Q_12. (This seems an inverse of @moralejo’s explanation…)
It’s not strange that this R is not unity because q_12 and Q_12 are different due to difference in pulse shape between FF and cosmics.
So, the value R = 0.947 which @rlopezcoto found in HG seems reasonable.
(Because pulse shape difference between FF and cosmics is larger in HG than LG, it’s also understandable that we only see the clear difference in HG.)

If I calculate the same R using average pulse shapes, the result is R = q_12/Q_12 ~ [0.96 (HG), ~0.98 (LG)].
So I would say my calculation seems consistent with Ruben’s with ~2% accuracy.
The small difference may come from the issues @moralejo mentioned, such as waveform analysis, event selection and PMT characteristics.

Any feedback is very welcome.

Let me put some relevant charge values (normalized by 20 ns integration) from my pulse shape analysis just for reference. (The two values mean HG and LG respectively)
Q8 = [0.84, 0.92]
Q12 =[0.95, 0.94]
q8 = [0.77, 0.86]
q12 = [0.91, 0.92]

[moralejo]

HI @yukihok, thanks for your input. You are right, I made a mistake and inverted the ratio obtained by @rlopezcoto. I edited my entry to avoid confusion.

@rlopezcoto I think this means that, contrary to what we had discussed, we do not have to incorporate any correction factor relative to what @FrancaCassol had calculated for use with 8 samples. The factor 1.056 you applied to the 8-sample integration here: https://user-images.githubusercontent.com/17042312/94912854-8f4ae880-04a8-11eb-9e00-ea76b4ab9663.png should actually be applied (inverted) to the 12-sample integration.

[rlopezcoto]

thanks @moralejo, @yukihok and @FrancaCassol for the explanations. It seems that the misleading thing to me was not to consider that the correction proposed was also dependent from Q_8 and therefore cancelled later when the ratio in the above shown plots was taken. I'll close #530