Add Cat-B calibration onsite scripts #1147
Conversation
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #1147 +/- ##
==========================================
+ Coverage 73.97% 74.42% +0.44%
==========================================
Files 124 129 +5
Lines 12647 13169 +522
==========================================
+ Hits 9356 9801 +445
- Misses 3291 3368 +77
☔ View full report in Codecov by Sentry. |
| @@ -77,7 +77,8 @@ class CalibrationCalculator(Component): | |||
|
|
|||
| hg_lg_ratio = traits.Float( | |||
| 1., | |||
| help='HG/LG ratio applied if use_scaled_low_gain is True. In case of calibrated data the ratio should be 1.' | |||
| help='HG/LG ratio applied if use_scaled_low_gain is True. The ratio is ~1 for calibrated data, ~17.4 for uncalibrated data.' | |||
|
|
|||
There was a problem hiding this comment.
Can we maybe change this into two options?
The hg_lg_ratio should be fixed (at least for LST), to 17.4.
It only needs to be applied for Cat-A.
So maybe it's simpler to leave that as 17.4 and have a switch calibrated_input or simular, that is set to False by the cat A calibration script but to True by the cat B script?
Or even automatically determined by the input / if an existing calibration file has been given?
There was a problem hiding this comment.
We never used this option for the Cat-A, it is instead useful for Cat-B calibration because pixels with low HV (due to high NSB) can not be correctly calibrated with LG. For the status of the code/data now, still in test phase, it seems the me that such a trailet is enough to deal with all cases (by experts). I do not see the necessity of an automatisation at this level (but a test can be eventually added at a higher level).
lstchain/scripts/onsite/onsite_create_cat_B_calibration_files_with_batch.py
Outdated
Show resolved
Hide resolved
| # if no events in dl2, e.g. for bad time interval from Cat-B calibration | ||
| if len(dl2) == 0: | ||
| logger.warning("No events in dl2. Return.") | ||
| return dl2 |
There was a problem hiding this comment.
+1 to not write the empty DL2 file
lstchain/scripts/onsite/onsite_create_cat_B_calibration_file.py
Outdated
Show resolved
Hide resolved
lstchain/scripts/onsite/onsite_create_cat_B_calibration_file.py
Outdated
Show resolved
Hide resolved
lstchain/scripts/onsite/onsite_create_cat_B_calibration_files_with_batch.py
Show resolved
Hide resolved
| # set unusable all pixels in periods with too many unusable pixels (car flashes, etc...) | ||
| catB_unusable_pixels[np.sum(catB_unusable_pixels, axis=2) > args.max_unusable_pixels] = True | ||
| # add good time interval column (gti) | ||
| catB_calib['gti'] = np.max(np.sum(catB_unusable_pixels, axis=2),axis=1) < args.max_unusable_pixels |
There was a problem hiding this comment.
A GTI is a good time interval. However, this is more a flag that this particular monitoring point "is valid" or has a general problem.
You would build the DL3 GTI by looking at all possible sources of issues (including the calibration monitoring data), but I wouldn't call this GTI here. Maybe just "is_valid"? We have xy_is_valid in many places in the ctapipe data structures.
There was a problem hiding this comment.
This is defining periods (times intervals) in which the camera is ok from the calibration point of view (all included), honestly I find that GTI is a clearer explicative naming for that (given also that is in the calibration table, so clearly only related to the calibration). "is_valid" seems to me less explicative. Is there a document where this naming is defined?
|
Hi, I would like to finalize this PR, are there other comments? |
Co-authored-by: Daniel Morcuende <[email protected]>
|
|
||
| Returns | ||
| ------- | ||
| sel_events: selected events |
There was a problem hiding this comment.
| sel_events: selected events | |
| pandas DataFrame or astropy.table.QTable | |
| Selected events |
|
Hi @rlopezcoto , @maxnoe, |
The scripts call the tool
lstchain_create_cat_B_calibration_fileand are equivalent to the Cat-A onsite scripts. They produce by default the Cat-B calibration files in the calibration data tree:/fefs/aswg/data/real/monitoring/PixelCalibration/Cat-B/calibration/Each calibration file will contain several calibration events, based on the interleaved events and relative to the Cat-A calibration. The files can then be used in the dl1a_to_dl1b process to correct the calibration during the night.