Add time calibration script

[FrancaCassol]

I add the script to calculate the time calibration which was missing by mistake in the PR #248

[rlopezcoto]

thanks @FrancaCassol! Just a small comment

[rlopezcoto]

is this the default number of events that you are using? Or are you using None to read them all?

[FrancaCassol]

It is the default value, I guess @pawel21 suggested it because it is enough to get a good estimation

[rlopezcoto]

ok, thanks, @pawel21 could you confirm that this is enough to get this accepted?

[pawel21]

@rlopezcoto , Yes, this should be enough to fill all bins in relative short time.

[FrancaCassol]

Hi @rlopezcoto, can we merge this?
Otherwise the calibration onsite script is not working

[morcuended]

I tried to run the script with the default config (max_events=20000) for several of the last runs. I got this error message.

...
    raise RuntimeError("Not enough events to coverage all capacitor. "
RuntimeError: Not enough events to coverage all capacitor. Please use more events to time calibration file.

[pawel21]

Dear @morcuended , I have checked this data, seems that calibration pulse in few pixels are missing, so script raise error. You can use data from November/December 2019.

[FrancaCassol]

Dear @pawel21, all,
I think this must be more strictly defined.

I am afraid it will often happen that one or more pixels are off, we can not leave to the shifter the search for an old time calibration file. If the pixels are off why should not the script work anyway for the other ones?

More generally speaking: How often must the time calibration coefficients be calculated? If the calculation is daily, the script must be robust enough to work with all pixel conditions, instead if we can calculate them on periodically good selected runs, I prefer to take it out from the onsite script

[moralejo]

Indeed, if the problem is that some pixel(s) had no signal, the error message is not correct. All pixels that can be calibrated should be calibrated and just a warning should be issued. This should be fixed regardless of how often we will in the end perform such a calibration - if only because in order to decide that, we will need initially to do it for every available calibration file, and check the stability of the corrections, and for that we need the code to be robust against missing pixels.

…

On Fri, 17 Jan 2020 at 09:17, Franca Cassol ***@***.***> wrote: Dear @pawel21 <https://github.com/pawel21>, all, I think this must be more strictly defined. I am afraid it will often happen that one or more pixels are off, we can not leave to the shifter the search for an old time calibration file. If the pixels are off why should not the script work anyway for the other ones? More generally speaking: How often must the time calibration coefficients be calculated? If the calculation is daily, the script must be robust enough to work with all pixel conditions, instead if we can calculate them on periodically good selected runs, I prefer to take it out from the onsite script — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#261?email_source=notifications&email_token=ADMCCOSD2ZGU7XXGH3E3O2TQ6FSQRA5CNFSM4KHQKYS2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJG4VLI#issuecomment-575523501>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADMCCOXVZR6E5G2WYKLJ5U3Q6FSQRANCNFSM4KHQKYSQ> .

-- Abelardo Moralejo Olaizola Institut de Física d'Altes Energies Tel : +34 931641662 Fax: +34 935811938 Avís - Aviso - Legal Notice - (LOPD) - http://legal.ifae.es

-- Avís - Aviso - Legal Notice - (LOPD) - http://legal.ifae.es <http://legal.ifae.es/>

[pawel21]

Dear Franca,
I think we don't need calcurate calibration coefficients every night. In MAGIC we use one time calibrattion file for 2 month or more. Offline, I will check stability of corrections.

[FrancaCassol]

I see.

@moralejo, we should then decide which protocol to follow to create these files: how often they must be created and where they must be stored so that the analysis scripts find them automatically. Then, somebody should be officially in charge for their periodical update

@pawel21, in any case, please make the script robust with respect to not working pixels.

[moralejo]

On Fri, 17 Jan 2020 at 10:36, Franca Cassol ***@***.***> wrote: @moralejo <https://github.com/moralejo>, we should then decide which protocol to follow to create these files: how often they must be created and where they must be stored so that the analysis scripts find them automatically. Then, somebody should be officially in charge for their periodical update

I think this is not a very time consuming addition to baseline calibration (Pawel, correct me if I am wrong). So, for simplicity I propose to keep it as part of the standard calibration script, and decide later (empirically) how often we want to update it.

@pawel21 <https://github.com/pawel21>, in any case, please make the script robust with respect to not working pixels.

@pawel21 <https://github.com/pawel21>, when do you think the modifications to distinguish between non-working pixels and sparse DRS4 coverage can be done? If not possible today (let us know), let's not delay the lstchain release. We can use the old file Pawel suggested in the meantime (I understand this would only affect the time calibration and not the other DRS4 calibrations, correct?).

…

-- Abelardo Moralejo Olaizola Institut de Física d'Altes Energies Tel : +34 931641662 Fax: +34 935811938 Avís - Aviso - Legal Notice - (LOPD) - http://legal.ifae.es

-- Avís - Aviso - Legal Notice - (LOPD) - http://legal.ifae.es <http://legal.ifae.es/>

[pawel21]

Dear @moralejo ,
Today is not possible to prepare such script, I will do it in next week. I am sorry.
This affect only time correction, not another drs4 correction.

[FrancaCassol]

@pawel21 https://github.com/pawel21, when do you think the modifications to distinguish between non-working pixels and sparse DRS4 coverage can be done? If not possible today (let us know), let's not delay the lstchain release. We can use the old file Pawel suggested in the meantime (I understand this would only affect the time calibration and not the other DRS4 calibrations, correct?).
…

Could we then define where must be located this "service" calibration file in the data tree?

├── real
│   │
│   │   ├── calibration
│   │   │       └── <date>
│   │   │           └── prod_ID

Should we create a svc calibration subdirectory ? @vuillaut do you have any suggestion?

@pawel could you please provide a recent good (full camera) time calibration file?

[pawel21]

I am sorry, . I don't have access to a computer until Sunday. @FrancaCassol could you produce time calibration file using data from run 1625 (2019/11/24, example notebook time_calibration) I would be grateful. Let me know in case of any problems.

[FrancaCassol]

Hi @pawel21,
yes, I can do it, but we need to decide where to put.