About Convert ADNI to BIDS Question

[wsxal]

Dear Dr. Lozupone,

I am an incoming graduate student specializing in Computer Science and Technology, with a focus on Computer Vision. Recently, I have been working on reproducing your AXIAL project and studying your paper titled "AXIAL: Attention-based eXplainability for Interpretable Alzheimer's Localized Diagnosis using 2D CNNs on 3D MRI brain scans." However, I have encountered the following issue and would greatly appreciate your guidance.

During the "Convert ADNI to BIDS" step, I ran the command clinica convert adni-to-bids -m T1 DATASET_DIRECTORY CLINICAL_DATA_DIRECTORY BIDS_DIRECTORY. Unfortunately, at the "Creating participants.tsv..." stage, I encountered the following error:

"ADNI", clinic_specs_path, clinical_data_dir, bids_ids
  File "/home/yang/.micromamba/envs/clinicaEnv/lib/python3.10/site-packages/clinica/iotools/bids_utils.py", line 84, in create_participants_df
    value_to_read = file_to_read[participant_fields_db[i]]
  File "/home/yang/.micromamba/envs/clinicaEnv/lib/python3.10/site-packages/pandas/core/frame.py", line 3458, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/yang/.micromamba/envs/clinicaEnv/lib/python3.10/site-packages/pandas/core/indexes/base.py", line 3363, in get_loc
    raise KeyError(key) from err
KeyError: 'APGEN1'

From the error log, it seems that the clinical data is missing a column named 'APGEN1'. However, I followed the instructions in the README file meticulously during the preceding download steps. I suspect that the issue might be due to an incompatible version of the Clinica tool. I have tried using some different versions of Clinica, but the problem persists.

For your reference, I have detailed the steps I took to download the clinical data below:

The version of clinica is 0.8.3

Could you please provide some advice on how to resolve this issue? Additionally, could you share the exact version of the Clinica tool that you used for your project?
Thank you very much for your time and assistance. I greatly appreciate your help and am looking forward to your response.

[GabrieleLozupone]

I've used clinica 0.7.6.

The errors come from create_participants_df function defined in this file: https://github.com/aramis-lab/clinica/blob/dev/clinica/iotools/bids_utils.py#L330.

There are differences in this file between 0.7.6 and 0.8.3 in lines 371-374:

In version 0.8.3:
elif file_ext == ".csv": file_to_read = load_clinical_csv( clinical_data_dir, location.split(".")[0] )
In version 0.7.6:
elif file_ext == ".csv": file_to_read = pd.read_csv(file_to_read_path)

You can find the new load_clinical_csv function there: https://github.com/aramis-lab/clinica/blob/dev/clinica/iotools/converters/adni_to_bids/adni_utils.py#L1102.

According to the function description:
This function is able to find the csv in the different known format available, the old format with just the name, and the new format with the name and the date of download.

So you can try to skip this part of the README file:

Rename CSV Files

Some CSV files in the clinical data may have a date at the end of their name. Remove the date from the file names to ensure >compatibility with the preprocessing pipeline.

Moreover, create_participants_df read participant_fields_db from the file partecipant.tsv.

If you replicate the clinica code, this the content of participant_fields_db:

0                    NaN
1                   PTID
2                    NaN
3                    NaN
4                    NaN
5                    NaN
6               PTGENDER
7               PTEDUCAT
8                    NaN
9                    AGE
10               PTMARRY
11              PTETHCAT
12                   RID
13                  SITE
14              ORIGPROT
15                   NaN
16                 DX_bl
17               MMSE_bl
18             ADAS11_bl
19             ADAS13_bl
20                 APOE4
21                APGEN1
22                APGEN2
23    RAVLT_immediate_bl
24               MOCA_bl
25           TRABSCOR_bl
26         mPACCdigit_bl
27       mPACCtrailsB_bl
Name: ADNI, dtype: object

So, the column named 'APGEN1' is not directly related to clinical data but needs to be present in partecipant.tsv and the file in clinical data in which the information is loaded is APOERES.csv.

I hope this information is helpful to you in resolving your issue. I'm sorry, but at the moment, I don't have the possibility to replicate your issue.

[diaoyq121]

I'm also experiencing this problem, have you solved it yet?。

[GabrieleLozupone]

I've tried to figure out what is the problem. According to the Clinica GitHub issues section Bug APGEN1, it is a bug due to the ADNI 4 update. The ADNI team changed the file structure with the latest dataset update, and the Clinica conversion tool needs to be updated accordingly.

I've tried to solve the issue by myself using the Clinica source code, but I have not yet found a solution.

[GabrieleLozupone]

I've found a way to complete the conversion without errors. I've posted a comment with the procedure here: Solution for ADNI to BIDS Conversion Issue