Unable to allocate

[animesh]

Describe the bug
Trying to run alpha-DIA phospho-enriched of 3 samples using timsTOF-pro using version 1.8.1

but facing following error

9 days, 5:25:09.773994 �ERROR: Search for zr_IMAC_300ug_zoom20_1dia_25pepsep_S1-A8_1_8451 failed with error: Unable to allocate 2.21 GiB for an array with shape (46, 12921749) and data type float32�
Traceback (most recent call last):
  File "alphadia\planning.py", line 342, in run
    psm_df, frag_df = workflow.extraction()
                      ^^^^^^^^^^^^^^^^^^^^^
  File "alphadia\workflow\peptidecentric.py", line 838, in extraction
    features_df, fragments_df = self.extract_batch(
                                ^^^^^^^^^^^^^^^^^^^
  File "alphadia\workflow\peptidecentric.py", line 816, in extract_batch
    features_df, fragments_df = candidate_scoring(
                                ^^^^^^^^^^^^^^^^^^
  File "alphadia\plexscoring.py", line 1915, in __call__
    candidate_features_df = self.collect_candidates(candidates_df, psm_proto_df)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "alphadia\plexscoring.py", line 1768, in collect_candidates
    df = utils.merge_missing_columns(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "alphadia\utils.py", line 683, in merge_missing_columns
    return left_df.merge(right_df[on + missing_from_left], on=on, how=how)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

To Reproduce
Steps to reproduce the behavior:

1. Go to 'Input files' and add data
   

2. Set up parameters
   

3. Click on 'Run workflow'

Logs
Please provide the log (see the AlphaDIA terminal on where to find it).

Method Setup
CPU: 2.69%
RAM: 26.90 GB / 127.92 GB (21.03%)
3 days, 5:52:24.788294 INFO: calibration group: precursor, fitting mz estimator 
3 days, 5:52:24.890499 INFO: calibration group: precursor, fitting rt estimator 
3 days, 5:52:24.989609 INFO: calibration group: precursor, fitting mobility estimator 
3 days, 5:52:25.088798 INFO: calibration group: fragment, fitting mz estimator 
3 days, 5:52:25.436969 INFO: calibration group: precursor, predicting mz
3 days, 5:52:25.469558 INFO: calibration group: precursor, predicting rt
3 days, 5:52:25.556145 INFO: calibration group: precursor, predicting mobility
3 days, 5:52:25.588838 INFO: calibration group: fragment, predicting mz
3 days, 5:52:25.946824 INFO: === checking if optimization conditions were reached ===
3 days, 5:52:25.947829 �PROGRESS: === Optimization of mobility_error has been performed 3 time(s); minimum number is 2 ===�
3 days, 5:52:25.951378 �PROGRESS: \u2705 mobility_error : optimization complete. Optimal parameter 0.0751 found after 3 searches.�
3 days, 5:52:25.951378 INFO: ==============================================
3 days, 5:52:25.952382 �PROGRESS: Optimization finished for mobility_error.�
3 days, 5:52:26.261122 INFO: calibration group: precursor, predicting mz
3 days, 5:52:26.276639 INFO: calibration group: precursor, predicting rt
3 days, 5:52:26.315721 INFO: calibration group: precursor, predicting mobility
3 days, 5:52:26.330461 INFO: calibration group: fragment, predicting mz
3 days, 5:52:26.531036 �PROGRESS: Search parameter optimization finished. Values taken forward for search are:�
3 days, 5:52:26.532031 �PROGRESS: ==============================================�
3 days, 5:52:26.532031 �PROGRESS: ms2_error      : 10.0000�
3 days, 5:52:26.532031 �PROGRESS: ms1_error      : 5.0000�
3 days, 5:52:26.533010 �PROGRESS: rt_error       : 697.5405�
3 days, 5:52:26.533010 �PROGRESS: mobility_error : 0.0751�
3 days, 5:52:26.534011 �PROGRESS: ==============================================�
3 days, 5:52:26.550471 INFO: calibration group: precursor, predicting mz
3 days, 5:52:29.037010 INFO: calibration group: precursor, predicting rt
3 days, 5:52:35.841354 INFO: calibration group: precursor, predicting mobility
3 days, 5:52:38.326065 INFO: calibration group: fragment, predicting mz
3 days, 5:53:06.057718 �PROGRESS: Extracting batch of 8106263 precursors�
3 days, 5:53:08.930254 INFO: Duty cycle consists of 11 frames, 1.17 seconds cycle time
3 days, 5:53:08.930254 INFO: Duty cycle consists of 928 scans, 0.00082 1/K_0 resolution
3 days, 5:53:08.930254 INFO: FWHM in RT is 5.92 seconds, sigma is 1.07
3 days, 5:53:08.931255 INFO: FWHM in mobility is 0.007 1/K_0, sigma is 3.87
3 days, 5:53:08.937792 INFO: Starting candidate selection

  0%|          | 0/1000 [00:00<?, ?it/s]
  0%|          | 1/1000 [00:00<02:16,  7.33it/s]
  0%|          | 2/1000 [07:10<70:08:58, 253.05s/it]
  0%|          | 3/1000 [14:43<95:27:31, 344.69s/it]
  0%|          | 4/1000 [22:35<109:10:50, 394.63s/it]
  0%|          | 5/1000 [30:26<116:42:03, 422.23s/it]
  1%|          | 6/1000 [38:16<121:03:11, 438.42s/it]
  1%|          | 7/1000 [46:08<123:57:18, 449.38s/it]
  1%|          | 8/1000 [54:03<126:04:48, 457.55s/it]
  1%|          | 9/1000 [1:01:50<126:49:38, 460.72s/it]
  1%|1         | 10/1000 [1:09:47<128:00:42, 465.50s/it]
  1%|1         | 11/1000 [1:17:17<126:38:55, 461.01s/it]
  1%|1         | 12/1000 [1:24:27<123:52:59, 451.40s/it]
  1%|1         | 13/1000 [1:31:40<122:13:56, 445.83s/it]
  1%|1         | 14/1000 [1:38:51<120:54:36, 441.46s/it]
  2%|1         | 15/1000 [1:45:59<119:40:16, 437.38s/it]
  2%|1         | 16/1000 [1:53:09<118:53:53, 434.99s/it]
  2%|1         | 17/1000 [2:00:19<118:24:07, 433.62s/it]
  2%|1         | 18/19 days, 1:38:41.329861 INFO: Applying score cutoff of 18.306091209411623
9 days, 1:38:42.386940 INFO: Removed 1473214 precursors with score below cutoff
9 days, 1:38:53.946519 INFO: Starting candidate scoring

  0%|          | 0/1000 [00:00<?, ?it/s]
  0%|          | 2/1000 [00:04<39:30,  2.38s/it]
  0%|          | 3/1000 [00:10<59:49,  3.60s/it]
  0%|          | 4/1000 [00:16<1:14:32,  4.49s/it]
  0%|          | 5/1000 [00:22<1:23:21,  5.03s/it]
  1%|          | 6/1000 [00:28<1:28:15,  5.33s/it]
  1%|          | 7/1000 [00:33<1:31:21,  5.52s/it]
  1%|          | 8/1000 [00:39<1:33:30,  5.66s/it]
  1%|          | 9/1000 [00:45<1:34:52,  5.74s/it]
  1%|1         | 10/1000 [00:51<1:36:05,  5.82s/it]
  1%|1         | 11/1000 [00:57<1:35:35,  5.80s/it]
  1%|1         | 12/1000 [01:02<1:30:42,  5.51s/it]
  1%|1         | 13/1000 [01:06<1:25:55,  5.22s/it]
  1%|1         | 14/1000 [01:11<1:22:56,  5.05s/it]
  2%|1         | 15/1000 [01:16<1:20:56,  4.93s/it]
  2%|1         | 16/1000 [01:20<1:19:03,  4.82s/it]
  2%|1         | 17/1000 [01:25<1:17:31,  4.73s/it]
  2%|1         | 18/1000 [01:29<1:16:47,  4.69s/it]
  2%|1         | 19/1000 [01:34<1:16:35,  4.68s/it]
  2%|2         | 20/1000 [01:39<1:9 days, 5:24:44.546288 INFO: Finished candidate processing
9 days, 5:24:44.548306 INFO: Collecting candidate features
9 days, 5:25:09.773994 �ERROR: Search for zr_IMAC_300ug_zoom20_1dia_25pepsep_S1-A8_1_8451 failed with error: Unable to allocate 2.21 GiB for an array with shape (46, 12921749) and data type float32�
Traceback (most recent call last):
  File "alphadia\planning.py", line 342, in run
    psm_df, frag_df = workflow.extraction()
                      ^^^^^^^^^^^^^^^^^^^^^
  File "alphadia\workflow\peptidecentric.py", line 838, in extraction
    features_df, fragments_df = self.extract_batch(
                                ^^^^^^^^^^^^^^^^^^^
  File "alphadia\workflow\peptidecentric.py", line 816, in extract_batch
    features_df, fragments_df = candidate_scoring(
                                ^^^^^^^^^^^^^^^^^^
  File "alphadia\plexscoring.py", line 1915, in __call__
    candidate_features_df = self.collect_candidates(candidates_df, psm_proto_df)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "alphadia\plexscoring.py", line 1768, in collect_candidates
    df = utils.merge_missing_columns(
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "alphadia\utils.py", line 683, in merge_missing_columns
    return left_df.merge(right_df[on + missing_from_left], on=on, how=how)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pandas\core\frame.py", line 10832, in merge
  File "pandas\core\reshape\merge.py", line 184, in merge
  File "pandas\core\reshape\merge.py", line 888, in get_result
  File "pandas\core\reshape\merge.py", line 879, in _reindex_and_concat
  File "pandas\core\reshape\concat.py", line 395, in concat
  File "pandas\core\reshape\concat.py", line 684, in get_result
  File "pandas\core\internals\concat.py", line 131, in concatenate_managers
  File "pandas\core\internals\concat.py", line 230, in _maybe_reindex_columns_na_proxy

Version (please complete the following information):

• Installation Type : One-Click Installer
  • Platform information
    • system Windows 11
    • machine x86_64
    • cpu count 24
  • Python information:
    • alphadia version 1.8.1

Additional context
phoSTY mods

[GeorgWa]

Hi,
This seems to be a memory issue. We currently only have limited support for searching large libraries with timsTOF.
How many files were processed before alphaDIA crashed?

There a re a couple of things worth trying:

• set the number of candidates in the Search tab to 1
• reduce the missed cleavages to 1

Some other observatios which should not affect memory but search speed:

• The number of threads looks relatively low for 24 cores. Maybe something like 20 would be better
• The mass accuracy is a bit low for timsTOF data. I would recomend 15ppm on the MS1 and MS2 level.

Best,
Georg

[animesh]

Thanks @GeorgWa for the suggestions 💯 There are 3 input files and I must note that i appended ;Phospho(STY) to the Mod-list, is that the correct way to go about it?

[GeorgWa]

Yes, the configuration looks correct.

You can find an overview of all modifications here
I think Phospho@S;Phospho@S;Phospho@Y should be the correct mod definition.

Can you also try to not set Acetyl and MethionineOX as variable modification?