Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug fix for the replacement of dl1 suffix #413

Merged
merged 4 commits into from
Jul 1, 2020

Conversation

SeiyaNozaki
Copy link
Collaborator

When using r0_to_dl1_filename, it seems another segment is removed by using with_suffix .

>>> from lstchain.paths import r0_to_dl1_filename
/fefs/aswg/workspace/seiya.nozaki/Install/anaconda3/envs/lst-dev/lib/python3.7/site-packages/corsikaio/subblocks/dtypes.py:20: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  return np.dtype(dict(**dt))
>>> r0_to_dl1_filename("/fefs/onsite/data/20200117/LST-1.1.Run01817.0000.fits.fz")
PosixPath('/fefs/onsite/data/20200117/dl1_LST-1.1.Run01817.h5')
>>> r0_to_dl1_filename("/fefs/aswg/workspace/MC_common/corsika6.9_simtelarray_2018-11-07/LST4_monotrigger/prod3/gamma/gamma_offaxis0.4deg_20190415/North_pointing/Data/gamma_20deg_0deg_run1___cta-prod3-demo-2147m-LaPalma-baseline-mono_off0.4.simtel.gz")
PosixPath('/fefs/aswg/workspace/MC_common/corsika6.9_simtelarray_2018-11-07/LST4_monotrigger/prod3/gamma/gamma_offaxis0.4deg_20190415/North_pointing/Data/dl1_gamma_20deg_0deg_run1___cta-prod3-demo-2147m-LaPalma-baseline-mono_off0.h5')

Here I just replaced with_suffix with +.

@codecov
Copy link

codecov bot commented May 20, 2020

Codecov Report

Merging #413 into master will increase coverage by 0.01%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #413      +/-   ##
==========================================
+ Coverage   41.74%   41.76%   +0.01%     
==========================================
  Files          77       77              
  Lines        6211     6213       +2     
==========================================
+ Hits         2593     2595       +2     
  Misses       3618     3618              
Impacted Files Coverage Δ
lstchain/scripts/lstchain_dl1_to_dl2.py 0.00% <ø> (ø)
lstchain/paths.py 90.62% <100.00%> (ø)
lstchain/tests/test_paths.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 65edd6a...27219da. Read the comment docs.

@rlopezcoto
Copy link
Contributor

Hi @SeiyaNozaki. I believe this may have been intended, take into account that the first file is a real data zfits file and the second one is a MC one and r0_to_dl1_filename should be dealing with R0 files.
Any comments @maxnoe ?

@maxnoe
Copy link
Member

maxnoe commented May 21, 2020

@rlopezcoto no, this is clearly a bug. Only the suffixes defined in EXTENSIONS_TO_REMOVE should be removed, everything else should stay intact.

So I would only suggest adding a test case for the file that was not correctly converted before @SeiyaNozaki

@SeiyaNozaki
Copy link
Collaborator Author

If file name includes dots before suffixes, the result is always bad.

>>> r0_to_dl1_filename("test.test.fits.fz")
PosixPath('dl1_test.h5')
>>> r0_to_dl1_filename("test.fits.fz")
PosixPath('dl1_test.h5')
>>> r0_to_dl1_filename("test.fits")
PosixPath('dl1_test.h5')
>>> r0_to_dl1_filename("test.test.fits")
PosixPath('dl1_test.h5')

So for the above case both of the results are bad(".Run01817." in real data, "0.4" in MC)

@maxnoe
Copy link
Member

maxnoe commented May 21, 2020

Yes, sure. You're fix is correct.

Could you add one example where the current code breaks to the tests?

@SeiyaNozaki
Copy link
Collaborator Author

@maxnoe ok, I added the examples, but I got the quality review error(Use of assert detected)...
How should I proceed?
(and I fixed some minor fix for other codes at the same time.)

Copy link
Contributor

@rlopezcoto rlopezcoto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got the quality review error(Use of assert detected)...How should I proceed?

@SeiyaNozaki it is alright

@maxnoe
Copy link
Member

maxnoe commented May 28, 2020

@maxnoe ok, I added the examples, but I got the quality review error(Use of assert detected)...
How should I proceed?

This codacy check unfortunately is not really reliable. Use of assert is required for unit tests, so it's ok. (Codacy should know that!)

@rlopezcoto rlopezcoto merged commit d373a68 into cta-observatory:master Jul 1, 2020
@SeiyaNozaki SeiyaNozaki deleted the dl1_suffix branch August 25, 2020 09:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants