endless OpenBLAS Warning #232
Comments
|
Hmm, that's not good. I've seen this once before but thought I was able to resolve it. I Was this in a fresh conda environment? What's your system? Linux, Mac? |
|
Thank you for your reply! I install concoct in a fresh environment, and I run it on CentOS. I tried to rebuild the library with USE_OPENMP=1 option, but it not work. It seems that the software could run and get bins. I tried a very small dataset. |
|
Yes I think this error could cause it to go slow and potentially the openmp is not working and thus it is not parallel at all. Maybe you should try to get this error to disappear befor making it run on a production data set? Could you please give me the output from:
From when you have activated the conda environment. Thank you! |
|
name: concoct
|
|
I can confirm I get the same error on our cluster with CentOS. You should probably cancel this run since it is likely to be extremely slow due to this error. I will look into why this is happening and come back to you. |
|
Waiting for your good news! ^_^ |
|
Ok, so not so good news. It seems like OpenMP is disabled in the conda available versions of OpenBLAS. I think I got around it by downgrading the openblas to the version before it was disabled. But in order to get it working I think you need to recompile concoct:
Please let me know if you manage to get this to work. If I can get this confirmed I can fix the version of openblas in the conda recipe as well. Thank you! |
|
Hi,
10 K contigs only need 45 min. |
|
That's a relief. I'll update the bioconda recipe accordingly. Thank you! |
|
I am unfortunately getting the same issue (repeating OpenBLAS Warning) in the metaWRAP conda environment: Please advise. |
|
I should also mention that I see this behavior ONLY when I use more than one thread. |
|
I'm not exactly sure what package that is triggering this error. You seem to have the correct version of |
|
Good question. I just installed it in a clear conda environment and still got the same issue, which is good because now there are less distracting factors: |
|
Ok, I agree, less distracting factors. I have to have a look at this right away. |
|
I am indeed able to reproduce this problem. I was however able to solve it locally by upgrading openblas to |
|
I think the problem is not the version of openblas. If you reinstall openblas after all packages installed, the problem will be solved. |
|
Hi @kodragonP, sorry I'm not following completely. Are you saying that if you reinstall the same version of openblas and still using conda, the problem is solved? You then have to uninstall both concoct and openblas first right? If you have the commands to reproduce this I would be very interested. |
|
I think I managed to fix the current bioconda recipe. @ursky, please could you verify that the latest build of concoct works for you? I've tested it on a fresh ubuntu container with python 2.7 and 3.7, and the '-t' parameter is now functional. |
|
That actually broke it, i think: |
|
The issue is that the |
|
I'm having the same problem. Should I manually configure the symlink as ursky suggested (and if so, can I literally copy paste those commands?) Or should I wait for concoct to finish and ignore the warnings? Or cancel the run and try again when the problem has been fixed? |
I think you should cancel it and try to configure the symlink. I had this problem last week, too. Then I updated metawrap to 1.18, and it worked.😂 |
But I met another problem that getting only one and big unbinning file when I ran CONCOCT by 'metawrap binning --CONCOCT'. Then I ran CONCOCT solely, and it was normal. I still don't understand. |
|
Can you give some details on the issue? Also, if concoct is already running and not exiting, you should be ok without restarting. |
|
And if the package is not found try searching for any mkl package. See all of the available ones with |
|
@ursky
Name Version Build Channelblas 1.0 mkl anaconda/cloud/conda-forge What should I do next？ Concoct works normal on the server in my lab. But it went error on the cluster in my institute. |
|
And if you run |
|
Hi @ursky
Can I run |
|
Looks like you have blas=2.7. So running |
|
@ursky Thank you~ it works |
|
I have the same problem: I use The used YAML file: channels:
- conda-forge
- defaults
- bioconda
dependencies:
- blas==2.10=mkl
- concoct==1.0.0
- maxbin2==2.2.6
- metabat2==2.13
- das_tool==1.1.1 |
|
Hi @ursky @alneberg . I reinstall the metawrap in my server. But the The following is the
|
|
I'm no expert on this but I THINK that I got mine to work. After reading through @ursky 's detective work, I did the following. It's running right now, I'll let you know if I get the error: # Move defaults to the top
conda config --add channels defaults
# Force the reinstall
conda install blas=2.5=mkl --force-reinstall |
|
Update, the above did'nt work. I got it to to run until completion on a small dataset but not for a larger. The following error message repeats:
Do we want |
|
@ursky Can you do |
|
Sorry for my lack of presence. I appreciate how much effort you're putting into this! Currently a fresh install of concoct through conda is affected of this problem, so I am currently attempting to fix this in the concoct conda recipe. I am hoping that removing openblas from the requirements of concoct (although it was added to solve exactly this issue to begin with) and also ensuring However, it seems like openblas has precedence over blas for linux so if conda-forge is the top ranked channel for conda, it will be installed when installing for example any numpy related package. So the resulting conda environment will be brittle, but hopefully this will at least make it work on a fresh install. |
|
Success! The current conda recipe creates a concoct env without this error. The solution was to remove openblas and enforce |
|
Just wanted to follow up with this. Has this issue been fixed? I kind of want to dive back into my dataset soon and there's LOTS of euks in there so I don't want to use metabat2 or maxbin2. I also don't want to run concoct with one thread haha. |
|
This doesnt answer the question but in my experience extracting eukaryotic MAGs, metabat2 usually performs the best. Id give it a try. |
|
@jolespin, I believe this issue was resolved yes. However, the nature of the problem makes it likely to come back. Did you try to create a fresh conda environment with concoct and check if you get the error? |
|
i dont have conda but pip. i tried with "pip install blas==2.5=mkl" and it doesn't work. |
|
FYI, I got the same issue in CentOS with the py310 version (default version of concoct=1.1.0) but not the py38 (installed by specifying python=3.8) |
|
I would like to point out that the OpenBLAS warning issue has reemerged with the most recent conda install only when trying to multithread. The above solutions unfortunately did not work for me when trying to fix the issue.
At this point you can try entering
At this point I was able to run concoct while multithreading without the repeated warning. |
Can confirm this fixes the issue for me as well. |
I installed concoct using conda, and when I run concoct main progress, endless
"OpenBLAS Warning : Detect OpenMP Loop and this application may hang. Please rebuild the library with USE_OPENMP=1 option. " would be printed. What happened?
The text was updated successfully, but these errors were encountered: