Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merge run wise and sorted #310

Merged
merged 3 commits into from
Mar 24, 2020
Merged

Merge run wise and sorted #310

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d973589
Allow merging files run-wise and sorted
morcuended Mar 12, 2020
68c6bc6
extend help comment for run-wise merging
morcuended Mar 24, 2020
fed768c
change to os path join and pep8-ing
morcuended Mar 24, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 16 additions & 3 deletions lstchain/scripts/lstchain_merge_hdf5_files.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,10 @@
from lstchain.io import get_dataset_keys
from lstchain.io import smart_merge_h5files, auto_merge_h5files

parser = argparse.ArgumentParser(description="Merge all HDF5 files resulting from parallel reconstructions \
present in a directory. Every dataset in the files must be readable with pandas.")
parser = argparse.ArgumentParser(description="Merge all HDF5 files resulting \
from parallel reconstructions present in a \
directory. Every dataset in the files must be \
readable with pandas.")


parser.add_argument('--source-dir', '-d', type=str,
Expand All @@ -29,12 +31,23 @@
help='Boolean. True to remove the images',
default=False)

parser.add_argument('--run-number', '-r', action='store', type=str,
dest='run_number',
help='Merge files run-wise if a run number is passed, \
otherwise merge all files in the directory',
default=None)

args = parser.parse_args()


def main():

file_list = [args.srcdir + '/' + f for f in os.listdir(args.srcdir) if f.endswith('.h5')]
if args.run_number:
file_list = sorted([os.path.join(args.srcdir, f) for f in os.listdir(args.srcdir)
if (f.endswith('.h5') and args.run_number in f)])
else:
file_list = sorted([os.path.join(args.srcdir, f) for f in os.listdir(args.srcdir)
if f.endswith('.h5')])

if args.noimage:
keys = get_dataset_keys(file_list[0])
Expand Down