Implement the dead time calculation #52
Conversation
|
So I implemented a function to calculate the dead time correction factor. The final correction factor is the multiplicity of those of LST-1 and MAGIC as suggested by @jsitarek in the issue #33. The factor for LST-1 is around 4%, and the one for MAGIC is around 1%, so in total it will be 5%. It would be great if @jsitarek could review it. Thank you in advance! |
There was a problem hiding this comment.
sorry for a late feedback. The deadtime calculation looks fine, but I put two sugestions to improve it further (to save us a headache in the future)
there seems to be also some other changes related to hot pixel finding and pedestals in the same PR, I guess those were not intended.
| condition = '(time_diff > 0) & (time_diff < 0.1)' | ||
|
|
||
| time_diffs_lst = df_events.query(f'(tel_id == 1) & {condition}')['time_diff'].to_numpy() | ||
| time_diffs_magic = df_events.query(f'(tel_id == 2) & {condition}')['time_diff'].to_numpy() |
There was a problem hiding this comment.
M1 and M2 dead time is strongly correlated so it is fine to take only one as you do, but maybe we can make it more "democratic", such that it would work also in the case we have observations when M1 was not working at all.
How about counting the M1 and M2 events and using for this one the one with more events?
| delta_time = time_end.value - time_start.value | ||
|
|
||
| deadc = effective_time / elapsed_time | ||
| elapsed_time = time_end.value - time_start.value |
There was a problem hiding this comment.
not sure over how big chunks of data this function can be run, but a safer way to compute the elapsed time is to take all the time difference, exclude big gaps (e.g. > 0.1 s), and sum them
|
Hi @jsitarek, thank you for your checks and comments! I removed the unrelated changes and implemented your suggestions. Regarding the elapsed time, I saw that the cut of the time diffs (< 0.1 sec) is too small for the LST-1 + MAGIC DL3 events. Please find the attached plot of the time difference distribution of DL3 events surviving gammaness cuts (dynamic, g/h efficiency = 0.9). It seems to me that the number of events reconstructed by the three telescopes is actually small, and in addition to this the superior gamma/hadron separation removes background events much better, and so gamma-ray events may already be dominant in the DL3 event list, which makes the time diffs bigger (than LST-1 mono and MAGIC-stereo). So at the moment I don't apply any time diff cuts when calculating the elapsed time, but please let me know what you think.
|
|
@jsitarek by the way it would be great if you could give an approval when you think it is ok to merge, thanks! |
There was a problem hiding this comment.
sorry for late feedback. I approve it if you want to merge it, but I have two remarks:
- concerning the calculation of the elapsed time - I see your point, this is actually very tricky to get properly if the rate of events is low. But I think it is unlikely that the rate will be dominated by gamma-rays: this would be the case around the source position, but not for the whole camera. If you keep all the events in one basket it should be fine, but if you start dividing into 4 types (following the RF) this indeed can be an issue. The current code with just adding the time differences look dangerous to me - it is quite likely to have some subruns gets missing in processing and then you have runs with holes in them what are not reflected in the effective time. We could use a larger time difference (10s maybe), or apply a fit and get the elapsed time from that fit, but those would require more tests, so it is fine for me to merge it as it is.
- github reports that the last line is missing the newline symbol, maybe you can check if it was not removed by mistake
|
Thanks @jsitarek, I added a new line at the end of the file. Regarding the elapsed time, you are right and I overstated that gamma-rays are dominant in a DL3 file. Let's make another pull request when the current code of the elapsed time calculation makes a problem. I will merge this to the master now. |
This pull request includes the dead time calculation in the combined analysis pipeline. It needs the newest version of ctapipe_io_magic to get the time differences of MAGIC events. It closes #33.